HARMLESS FRAME FILTER, HARMFUL IMAGE BLOCKING APPARATUS HAVING THE SAME, AND METHOD FOR FILTERING HARMLESS FRAMES

• US 20150221097A1
• Filed: 10/30/2014
• Published: 08/06/2015
• Est. Priority Date: 02/05/2014
• Status: Abandoned Application

• Current Assignee: Electronics and Telecommunications Research Institute
• Original Assignee: Electronics and Telecommunications Research Institute
• Inventors: YU, Jung-Jae, KIM, Moo-Seop, CHOI, Su-Gil, JEONG, Chi-Yoon, HAN, Seung-Wan
• Application Number:
  US14/529,120
• Publication Number:
  US 20150221097A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 2207/20036 Morphological image processing
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 20/00 Scenes; Scene-specific elements control of digital cameras H04N23/60
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands

Abstract

Disclosed herein are a harmless frame filter, a harmful image blocking apparatus having the filter, and a method for filtering harmless frames, which can rapidly identify clearly harmless images and exclude those harmless images in advance upon determining harmful image content including obscene images in real time, thus improving task efficiency. The harmless frame filter includes a primary filtering unit for extracting skin region candidate pixels from an input image, and primarily filtering harmless frames, based on a ratio of the skin region candidate pixels to the input image, and a secondary filtering unit for generating a Block Binary Pattern (BBP) histogram representing distribution characteristics of the skin region candidate pixels in a two-dimensional (2D) image space, and secondarily filtering harmless frames by comparing predefined learning data with the BBP histogram.

Claims

1. A harmless frame filter, comprising:

a primary filtering unit for extracting skin region candidate pixels from an input image, and primarily filtering harmless frames, based on a ratio of the skin region candidate pixels to the input image; and

a secondary filtering unit for generating a Block Binary Pattern (BBP) histogram representing distribution characteristics of the skin region candidate pixels in a two-dimensional (2D) image space, and secondarily filtering harmless frames by comparing predefined learning data with the BBP histogram.

2. The harmless frame filter of claim 1, wherein the primary filtering unit comprises a skin pixel extraction module for converting RGB color information at pixels of the input image into a Hue, Saturation, Value (HSV) color space to generate a hue histogram, and determining a maximum density region, in which a sum of densities is maximum on a predefined hue (H) channel of the hue histogram, to be a range of the skin region candidate pixels.

3. The harmless frame filter of claim 2, wherein the primary filtering unit further comprises a skin pixel-based primary filtering module configured to:

when the maximum density sum of the maximum density region is less than or equal to a first preset value, determine the input image to be a harmless image; and

when the maximum density sum is greater than the first preset value, determine the input image to be a harmful candidate image, generate a binary alpha map from the skin region candidate pixels, and transfer the binary alpha map to the secondary filtering unit.

4. The harmless frame filter of claim 3, wherein the secondary filtering unit comprises a skin region post-processing module for performing post-processing on the alpha map and then generating a post-processed alpha map in which noise effects are corrected.

5. The harmless frame filter of claim 4, wherein the skin region post-processing module comprises:

an edge image generation sub-module for converting a color input image into a gray image, and generating an edge image using an edge operator;

an edge sum image generation sub-module for generating an edge sum image;

an edge density image generation sub-module for generating edge densities;

an alpha map filtering sub-module for determining pixels, edge densities of which are equal to or greater than a second preset value in the alpha map, to be a background region, and modifying the alpha map; and

a morphological operation sub-module for reducing noise of the alpha map by sequentially applying morphological closing and opening operations of the edge operator.

6. The harmless frame filter of claim 4, wherein the secondary filtering unit further comprises a BBP histogram generation module for extracting, from the post-processed alpha map, distribution characteristics of skin pixels extracted from a human skin on an image plane, wherein the distribution characteristics of the skin pixels are differentiated from those of skin color pixels of a background region mistaken for the skin region due to similarity to a skin color.

7. The harmless frame filter of claim 6, wherein the BBP histogram generation module comprises:

an alpha map block matrix generation sub-module for dividing the post-processed alpha map into image blocks formed as a predetermined number of matrices, and classifying each division image block as a skin image block or a non-skin image block, based on a ratio of skin pixels to the division image block;

a BBP matching sub-module for representing, based on the skin image block, a distribution form of the skin image block with respect to neighboring image blocks by 51 non-overlapping BBPs, by means of a BBP block rotational transform and a BBP array direction inverse transform; and

a BBP histogram generation sub-module for detecting a number of BBPs represented by the BBP matching sub-module and then generating a BBP histogram.

8. The harmless frame filter of claim 7, wherein the BBP matching sub-module represents 51 BBPs within a range of an alpha map having division image blocks, each BBP being represented by a 3×

3 matrix in which an element corresponding to a second row and a second column is a skin image block.

9. The harmless frame filter of claim 1, wherein the secondary filtering unit comprises a BBP-based secondary filtering module for designating the BBP histogram as a feature vector having a spatial dimension of 51, and comparing the feature vector as the predefined learning data based on K-means clustering or a Support Vector Machine (SVM), thus secondarily filtering harmless frames.

10. A harmful image blocking apparatus, comprising:

a harmfulness determination unit for determining, for filtered input images obtained by filtering harmless frames using the harmless frame filter as set forth in claim 1 , whether the filtered input images are harmful images, based on learning data stored in a harmful content preliminary knowledge database (DB); and

a harmful image blocking unit for accumulating and aggregating results of determination performed by the harmfulness determination unit about whether the filtered input images are harmful images on a time axis, finally determining whether an image currently being played is a harmful image, and blocking the image if it is determined that the image is a harmful image.

11. A method of filtering harmless frames, comprising:

performing, by a primary filtering unit, primary filtering by extracting skin region candidate pixels from an input image, and by primarily filtering harmless frames, based on a ratio of the skin region candidate pixels to the input image; and

performing, by a secondary filtering unit, secondary filtering by generating a Block Binary Pattern (BBP) histogram representing distribution characteristics of the skin region candidate pixels in a two-dimensional (2D) image space, and by comparing predefined learning data with the BBP histogram to secondarily filter harmless frames.

12. The method of claim 11, wherein performing the primary filtering comprises performing skin pixel extraction by converting RGB color information at pixels of the input image into a Hue, Saturation, Value (HSV) color space to generate a hue histogram and by determining a maximum density region, in which a sum of densities is maximum on a predefined hue (H) channel of the hue histogram, to be a range of the skin region candidate pixels.

13. The method of claim 12, wherein performing the primary filtering further comprises, after performing the skin pixel extraction, performing skin pixel-based primary filtering by:

when the maximum density sum of the maximum density region is less than or equal to a first preset value, determining the input image to be a harmless image, and

when the maximum density sum is greater than the first preset value, determining the input image to be a harmful candidate image, and generating a binary alpha map from the skin region candidate pixels.

14. The method of claim 13, wherein performing the secondary filtering comprises performing skin region post-processing by performing post-processing on the generated alpha map, and then generating a post-processed alpha map in which noise effects are corrected.

15. The method of claim 14, wherein performing the skin region post-processing comprises:

converting a color input image into a gray image, and generating an edge image using an edge operator;

generating an edge sum image;

generating edge densities;

determining pixels, edge densities of which are equal to or greater than a second preset value in the alpha map, to be a background region, and then modifying the alpha map; and

reducing noise of the alpha map by sequentially applying morphological closing and opening operations of the edge operator.

16. The method of claim 14, wherein performing the secondary filtering further comprises, after performing the skin region post-processing:

performing BBP histogram generation by extracting, from the post-processed alpha map, distribution characteristics of skin pixels extracted from a human skin on an image plane, wherein the distribution characteristics of the skin pixels are differentiated from those of skin color pixels of a background region mistaken for the skin region due to similarity to a skin color.

17. The method of claim 16, wherein performing the BBP histogram generation comprises:

dividing the post-processed alpha map into image blocks formed as a predetermined number of matrices, and classifying each division image block as a skin image block or a non-skin image block, based on a ratio of skin pixels to the division image block;

representing, based on the skin image block, a distribution form of the skin image block with respect to neighboring image blocks by 51 non-overlapping BBPs, by means of a BBP block rotational transform and a BBP array direction inverse transform; and

detecting a number of represented BBPs and then generating a BBP histogram.

18. The method of claim 17, wherein representing the distribution form comprises representing 51 BBPs within a range of an alpha map having division image blocks, each BBP being represented by a 3×

3 matrix in which an element corresponding to a second row and a second column is a skin image block.

19. The method of claim 11, wherein performing the secondary filtering comprises, after generating the BBP histogram, designating the BBP histogram as a feature vector having a spatial dimension of 51, and comparing the feature vector as the predefined learning data based on K-means clustering or a Support Vector Machine (SVM), thus secondarily filtering harmless frames.

20. The method of claim 11, further comprising, after performing the secondary filtering:

determining, for filtered input images, obtained by performing the secondary filtering, whether the filtered input images are harmful images, based on learning data stored in a harmful content preliminary knowledge database (DB); and

accumulating and aggregating results of the determination about whether the filtered input images are harmful images on a time axis, finally determining whether an image currently being played is a harmful image, and blocking the image if it is determined that the image is a harmful image.