Apparatus and method for detecting opaque logos within digital video signals

• US 20050078222A1
• Filed: 10/09/2003
• Published: 04/14/2005
• Est. Priority Date: 10/09/2003
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Kim, Yeong-Taeg, Liu, Shan
• Granted Patent: US 7,483,484 B2
• Time in Patent Office: Days
• US Class Current: 348/700
• CPC Class Codes: G06V 20/635 Overlay text, e.g. embedded captions in a TV program
• CPC Class Codes: H04N 5/147 Scene change detection

Abstract

A detection method and system that allows detecting an opaque logo after it has appeared in the digital video program for a short period of time, by deriving and analyzing the stochastic characteristics of the video signal along the temporal axis.

Claims

1. A method of detecting an opaque image in an area of each of a sequence of frames comprising pixels that represent digital video images, comprising the steps of:

detecting a scene change between at least two of said frames, designated as a current frame and a reference frame;

upon detecting a scene change, calculating a stochastic measure σ

_d of a plurality of pixels in the image frame based on the standard deviation of differences in pixel values in said two frames in the temporal domain; and

detecting an opaque image in said area by comparing the stochastic measure σ

_d with a given noise level (STD).

2. The method of claim 1, wherein the steps of detecting a scene change further includes the steps of:

determining differences in pixel intensities between said two frames;

subtracting the noise mean from said differences;

comparing the absolute values of the differences with noise mean subtracted to a first threshold value to detect pixels with significant intensity changes; and

comparing a percentage of a plurality of changed pixels in the current frame to a second threshold value to detect a scene change.

3. The method of claim 2, wherein the steps of determining differences in pixel intensities further include the steps of determining absolute difference in pixel intensities between said two frames.

4. The method of claim 2, wherein the first threshold value t₁ is a function of noise in the video images.

5. The method of claim 4, wherein the first threshold value t₁ is determined as t₁=a×

n+b such that n denotes the noise level, and a and b are adjustable parameters wherein increasing the values of a and/or b results in lower scene change detection sensitivity.

6. The method of claim 1, wherein the steps of detecting a scene change further includes the steps of:

generating a count C of the number pixels in the current frame whose difference in intensity relative to corresponding pixels in the reference frame, after subtracting the given noise mean, exceeds the threshold value.

7. The method of claim 1, wherein the steps of detecting a scene change further includes the steps of, for each of a plurality of pixels in the current frame and each corresponding pixel in the reference frame:

determining the difference in pixel intensity between said two pixels;

subtracting the noise mean from said difference;

comparing the absolute value of the difference, from which the noise mean has been subtracted, to a first threshold value; and

incrementing a running count C if the difference exceeds the first threshold value.

8. The method of claim 6, wherein the steps of detecting a scene change further includes the steps of:

determining a value p as a function of the ratio of said count C and the total number N of pixels in each frame; and

comparing the value p to a second threshold value to detect scene change.

9. The method of claim 1 wherein the steps of calculating the stochastic measurement σ

_d further includes the steps of;

generating the squared differences obtained for each of a plurality of said pixels;

subtracting the noise mean from each difference;

generating the sum of the absolute values of said difference with noise mean subtracted; and

generating an average of the sums, and determining the stochastic measure σ

_d based on said average.

10. The method of claim 9, wherein the stochastic measurement σ

_d is determined as;

${\mathrm{\sigma <br><br>}}_d=\sqrt{\frac{\underset{i=1}{\overset{M}{\sum <br><br>}}<br><br>|d<br><br>\left(t\right)-{\mathrm{\mu <br><br>}}_s<br><br>{|}^2}{M}}$ $\mathrm{wherein}<br><br>\text{;<br><br>}$ $d<br><br>\left(t\right)=p<br><br>\left(t\right)-p<br><br>\left(t-\mathrm{\tau <br><br>}\right),$

M is the number of frames involved in scene change detection,

p(t) denotes the pixel intensity value in one of said M frames, and

p(t−

τ

) denotes the corresponding pixel intensity value in another said M frames.

11. The method of claim 1, wherein the step of detecting an opaque image further includes the steps of:

classifying each pixel in said area of the current frame as an opaque image pixel, if the σ

_d for that pixel matches the given noise level, represented by the standard deviation σ

_s.

12. The method of claim 11, wherein the step of detecting an opaque image further includes the steps of:

classifying a pixel p(i) in the current frame as an opaque image pixel, if

|σ

_d(i)−

σ

_s|<

t, otherwise, classifying the pixel p(i) as a non-opaque image pixel,

wherein,

p(i) is the i^th pixel in the current frame,

σ

_s is standard deviation of noise in the video Signal, and

t is a pre-defined threshold for tolerating measurement and computation errors.

13. The method of claim 11, further comprising the steps of:

detecting and reclassifying any mis-classified pixels in a refinement process.

14. The method of claim 13, wherein the refinement process further includes the steps of:

if a pixel is classified as a an opaque image pixel, while less than two of its neighboring pixels are classified as opaque image pixels, then reclassifying that pixel as a non-opaque image pixel; and

if a pixel is classified as a non-opaque image pixel, while more than two of its neighboring pixels are classified as opaque image pixels, then reclassifying that pixel as an opaque image pixel.

15. An opaque image detection system for detecting an opaque image in an area of each of a sequence of frames including pixels that represent digital video images, comprising:

a scene change detector that detects scene change between at least two of said frames, designated as a current frame and a reference frame;

a stochastic characteristic generator that upon detection of scene change by the scene change detector, calculates a stochastic measure σ

_d of said area based on the standard deviation of differences in pixel values in said two frames in the temporal domain; and

an opaque image detector that detects an opaque image in said area by comparing the stochastic measure σ

_d with a given noise level, represented by the standard deviation σ

_s.

16. The system of claim 15, wherein the scene change detector further determines differences in pixel intensities between said two frames, and compares the differences to a first threshold value to detect significantly changed pixels, and compares the percentage of significantly changed pixels within the current frame to a second threshold to detect a scene change.

17. The system of claim 16, wherein the scene change detector determines differences in pixel intensities by determining absolute difference in pixel intensities between said two frames.

18. The system of claim 16, wherein the first threshold value t₁ is a function of noise in the video images.

20. The system of claim 16, wherein the scene change detector further generates a count C of the number pixels in the current frame whose differences in intensities relative to corresponding pixels in the reference frame, with noise mean subtracted, exceeds the first threshold value.

21. The system of claim 16, wherein the scene change detector is further configured such that, for each of a plurality of pixels in the current frame and each corresponding pixel in the reference frame the scene change detector determines the difference in pixel intensity between said two pixels with noise mean subtracted, compares the difference to the first threshold value, and increments a running count C if the difference exceeds the first threshold value.

22. The system of claim 21, wherein the scene change detector further determines a value p as a function of the ratio of said count C and the total number N of pixels in each frame, and compares the value p to a second threshold value to detect scene change.

23. The system of claim 15, wherein the stochastic characteristic generator further calculates the stochastic measurement σ

_d by generating a sum of the squared differences, with noise mean subtracted, obtained for each of a plurality of said pixels, generating an average of the sums, and determining the stochastic measure σ

_d based on said average.

24. The system of claim 23, wherein the stochastic measurement σ

_d is determined as;

${\mathrm{\sigma <br><br>}}_d=\sqrt{\frac{\underset{i=1}{\overset{M}{\sum <br><br>}}<br><br>|d<br><br>\left(t\right)-{\mathrm{\mu <br><br>}}_s<br><br>{|}^2}{M}}$ $\mathrm{wherein}<br><br>\text{;<br><br>}$ $d<br><br>\left(t\right)=p<br><br>\left(t\right)-p<br><br>\left(t-\mathrm{\tau <br><br>}\right),$

M is the number of frames involved in scene change detection,

p(t) denotes the pixel intensity value in one of said M frames, and

p(t−

τ

) denotes the corresponding pixel intensity value in another said M frames.

25. The system of claim 15, wherein the opaque image detector further classifies each pixel in said area of the current frame as an opaque image pixel, if the σ

_d for that pixel matches a given noise level, represented by the standard deviation σ

_s.

26. The system of claim 25, wherein the opaque image detector further classifies a pixel p(i) in the current frame as an opaque image pixel, if |σ

_d(i)−

σ

_s|<

t, otherwise, the opaque image detector classifies the pixel p(i) as a non-opaque image pixel, wherein;

p(i) is the ith pixel in the current frame, and

σ

_s is standard deviation of noise in the video Signal, and

t is a pre-defined threshold for tolerating measurement and computation errors.

27. The system of claim 26, further comprising a refinement module that detects and reclassifies any mis-classified pixels in a refinement process.

28. The system of claim 27, wherein the refinement module is further configured such that if a pixel is classified as a an opaque image pixel, while less than two of its neighboring pixels are classified as opaque image pixels, then the refinement module reclassifies that pixel as a non-opaque image pixel, and if a pixel is classified as a non-opaque image pixel, while more than two of its neighboring pixels are classified as opaque image pixels, then the refinement module reclassifies that pixel as an opaque image pixel.

19. The system of claim 19, wherein the first threshold value t₁ is determined as t₁=a×

n+b such that n denotes the noise level, and a and b are adjustable parameters wherein increasing the values of a and/or b results in lower scene change detection sensitivity.