Methods and systems for detecting pictorial regions in digital images

• US 8,630,498 B2
• Filed: 06/15/2006
• Issued: 01/14/2014
• Est. Priority Date: 03/02/2006
• Status: Active Grant

• Current Assignee: Sharp Kabushiki Kaisha (Hon Hai Precision Industry Co., Ltd.)
• Original Assignee: Sharp Laboratories of America Incorporated (Hon Hai Precision Industry Co., Ltd.)
• Inventors: Campbell, Richard John, Ferman, Ahmet Mufit, Matsuda, Toyohisa, Chen, Lawrence Shao-hsien
• Primary Examiner(s): Vu, Kimyen
• Assistant Examiner(s): HU, FRED H.
• Application Number:
  US11/424,296
• Publication Number:
  US 20070206857A1
• Time in Patent Office: 2,770 Days
• Field of Search: 382/232
• US Class Current: 382/232
• CPC Class Codes: G06V 10/507 Summing image-intensity values; Histogram projection analysis
• CPC Class Codes: G06V 30/413 Classification of content, e.g. text, photographs or tables

Abstract

Embodiments of the present invention comprise systems, methods and devices for detection of pictorial regions in an image using a masking condition, an entropy measure, and region growing.

Claims

1. A method for detecting a pictorial region in a digital image, said method comprising:

forming a masked image of a digital image, wherein said masked image comprises a first plurality of pixel locations, whereat each pixel in said first plurality of pixel locations in said masked image is assigned a mask-pixel value, wherein said first plurality of pixel locations in said masked image corresponds to a first plurality of pixel locations in said digital image whereat a masking condition based on edge strength is satisfied; and

said masked image further comprises a second plurality of pixel locations, wherein said second plurality of pixel locations in said masked image corresponds to a second plurality of pixel locations in said digital image whereat said masking condition is not satisfied, and wherein each pixel in said second plurality of pixel locations in said masked image is assigned a value of said corresponding pixel in said digital image;

in a region-detection system comprising a calculator, calculating a masked entropy value for each pixel in a third plurality of pixels in said masked image, wherein said masked entropy value for said each pixel is based on a frequency-of-occurrence, in said masked image, of pixel values not equal to said mask-pixel value, in a region proximate to said each pixel, and a function of said frequency-of-occurrence, wherein said function comprises a mathematical characteristic similar to a logarithmic function;

determining, in said region-detection system, a confidence level for said each pixel in said third plurality of pixels in said masked image based on said masking condition;

determining, in said region-detection system, a seed region comprising a seed-region pixel, wherein said determining a seed region comprises selecting said seed-region pixel, from said third plurality of pixels in said masked image, when said masked entropy value calculated for said seed-region pixel is a reliable value, wherein said reliable value is a relatively high masked entropy value; and

in said region-detection system, growing said seed region based on said confidence levels thereby producing a pictorial region wherein said pictorial region comprises pictorial-region pixels from said third plurality of pixels in said masked image.

2. A method as described in claim 1 further comprising refining said pictorial region thereby producing a refined pictorial region.

3. A method as described in claim 2 wherein said refining comprises filling at least one of holes and concave boundary regions in said pictorial region.

4. A method as described in claim 1 further comprising verifying said pictorial region thereby producing a verified pictorial region.

5. A method as described in claim 4 wherein said verifying said pictorial region comprises determining at least one of the size of said pictorial region, the shape of said pictorial region, the area of said pictorial region within a first bounding shape, and the distribution of said pictorial region with a second bounding shape.

6. A method as described in claim 1 wherein said determining said seed region further comprises:

determining a first confidence-level threshold value; and

identifying a target pixel from said third plurality of pixels in said masked image as one of said seed-region pixels if said confidence level for said target pixel is greater than said first confidence-level threshold value.

7. A method as described in claim 1 wherein said growing further comprises:

determining a second confidence-level threshold value; and

identifying a candidate pixel from said third plurality of pixels in said masked image as a pictorial-region pixel if said candidate pixel is connected to said seed region and said confidence level for said candidate pixel is greater than said second confidence-level threshold value.

8. A method as described in claim 6 wherein said first confidence-level threshold is based on the range of said confidence levels.

9. A method as described in claim 7 further comprising:

obtaining a labeled background map for said digital image; and

identifying said candidate pixel from said third plurality of pixels in said masked image as a pictorial-region pixel if said candidate pixel is less than said second confidence-level threshold value and said candidate pixel is labeled as pictorial content in said labeled background map.

10. A system for detecting a pictorial region in a digital image, said system comprising:

a mask generator for forming a masked image of a digital image, wherein said masked image comprises a first plurality of pixel locations, whereat each pixel in said first plurality of pixel locations in said masked image is assigned a mask-pixel value, wherein said first plurality of pixel locations in said masked image corresponds to a first plurality of pixel locations in said digital image whereat a masking condition based on edge strength is satisfied; and

said masked image further comprises a second plurality of pixel locations, wherein said second plurality of pixel locations in said masked image corresponds to a second plurality of pixel locations in said digital image whereat said masking condition is not satisfied, and wherein each pixel in said second plurality of pixel locations in said masked image is assigned a value of said corresponding pixel in said digital image;

a calculator processor for calculating a masked entropy value for each pixel in a third plurality of pixels in said masked image, wherein said masked entropy value for said each pixel is based on a frequency-of-occurrence, in said masked image, of pixel values not equal to said mask-pixel value, in a region proximate to said each pixel, and a function of said frequency-of-occurrence, wherein said function comprises a mathematical characteristic similar to a logarithmic function;

a first determiner for determining a confidence level for said each pixel in said third plurality of pixels in said masked image based on said masking condition;

a second determiner for determining seed regions based on said masked entropy values, wherein said second determiner determines a first pixel is a seed-region pixel when a first masked entropy value calculated for said first pixel is considered a reliable value based on a relative greatness of said first masked entropy value in relation to a plurality of other masked entropy values calculated; and

a region grower for growing said regions based on said confidence levels.

11. A system as described in claim 10 further comprising a refiner for refining said pictorial region thereby producing a refined pictorial region.

12. A system as described in claim 11 wherein said refining comprises filling at least one of holes and concave boundary regions in said pictorial region.

13. A system as described in claim 10 further comprising a verifier for verifying said pictorial region thereby producing a verified pictorial region.

14. A system as described in claim 13 wherein said verifier comprises at least one of a size determiner for determining the size of said pictorial region, a shape determiner for determining the shape of said pictorial region, an area determiner for determining the area of said pictorial region within a first bounding shape, and a distribution determiner for determining the distribution of said pictorial region within a second bounding shape.

15. A system as described in claim 10 wherein said second determiner for determining said seed region further comprises:

a first-confidence-level-threshold determiner for determining a first confidence-level threshold value; and

a seed-region-pixel identifier for identifying a target pixel from said third plurality of pixels in said masked image as one of said seed-region pixels if said confidence level for said target pixel is greater than said first confidence-level threshold value.

16. A system as described in claim 10 wherein said growing further comprises:

a second-confidence-level-threshold determiner for determining a second confidence-level threshold value; and

a first pictorial-region-pixel identifier for identifying a candidate pixel from said third plurality of pixels in said masked image as a pictorial-region pixel if said candidate pixel is connected to said seed region and said confidence level for said candidate pixel is greater than said second confidence-level threshold value.

17. A system as described in claim 15 wherein said first confidence-level threshold is based on the range of said confidence levels.

18. A system as described in claim 16 further comprising:

an obtainer for obtaining a labeled background map for said digital image; and

a second pictorial-region-pixel identifier for identifying said candidate pixel from said third plurality of pixels in said masked image as a pictorial-region pixel if said candidate pixel is less than said second confidence-level threshold value and said candidate pixel is labeled as pictorial content in said labeled background map.

19. A method for detecting a pictorial region in a digital image, said method comprising:

forming a masked image of a digital image, wherein said masked image comprises a first plurality of pixel locations, whereat each pixel in said first plurality of pixel locations in said masked image is assigned a mask-pixel value, wherein said first plurality of pixel locations in said masked image corresponds to a first plurality of pixel locations in said digital image whereat a masking condition related to edge strength is satisfied; and

said masked image further comprises a second plurality of pixel locations, wherein said second plurality of pixel locations in said masked image corresponds to a second plurality of pixel locations in said digital image whereat said masking condition is not satisfied, and wherein each pixel in said second plurality of pixel locations in said masked image is assigned a value of said corresponding pixel in said digital image;

in a region-detection system comprising a calculator, calculating a masked entropy value for each pixel in a third plurality of pixels in said masked image, wherein said masked entropy value for said each pixel is based on a frequency-of-occurrence, in said masked image, of pixel values not equal to said mask-pixel value, in a region proximate to said each pixel, and a function of said frequency-of-occurrence, wherein said function comprises a mathematical characteristic similar to a logarithmic function;

in said region-detection system, determining a confidence level for said each pixel in said third plurality of pixels in said masked image based on said masking condition associated with said mask;

in said region-detection system, determining a seed region comprising a seed-region pixel, wherein said determining a seed region comprises selecting said seed-region pixel, from said third plurality of pixels in said masked image, when said masked entropy value calculated for said seed-region pixel is a reliable value, wherein a said reliable value is a relatively high masked entropy value;

in said region-detection system, obtaining a labeled background map for said digital image wherein said labeled background map comprises a label corresponding to pictorial content;

in said region-detection system, growing said seed region based on said confidence levels and said labeled background map thereby producing a pictorial region wherein said pictorial region comprises pictorial-region pixels from said third plurality of pixels in said masked image;

in said region-detection system, refining said pictorial region thereby producing a refined pictorial region; and

in said region-detection system, verifying said refined pictorial region thereby producing a verified pictorial region.

20. The method as described in claim 19 further comprising:

determining a first confidence-level threshold value;

identifying a target pixel from said third plurality of pixels in said masked image as one of said seed-region pixels if said confidence level for said target pixel is greater than said first confidence-level threshold value;

determining a second confidence-level threshold value;

identifying a candidate pixel from said third plurality of pixels in said masked image as a pictorial-region pixel if said candidate pixel is connected to said seed region and said confidence level for said candidate pixel is greater than said second confidence-level threshold value; and

identifying said candidate pixel from said third plurality of pixels in said masked image as a pictorial-region pixel if said candidate pixel is less than said second confidence-level threshold value and said candidate pixel is labeled as pictorial content in said labeled background map.