Method for tracking the disappearance of detected logos within digital video signals

• US 7,356,084 B2
• Filed: 10/09/2003
• Issued: 04/08/2008
• Est. Priority Date: 10/09/2003
• Status: Expired due to Fees

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Kim, Yeong-Taeg, Liu, Shan
• Primary Examiner(s): RAO, ANAND SHASHIKANT
• Application Number:
  US10/682,315
• Publication Number:
  US 20050078223A1
• Time in Patent Office: 1,643 Days
• Field of Search: 37524011-24029
• US Class Current: 375/240.26
• CPC Class Codes: G06V 20/62 Text, e.g. of license plates, overlay texts or captions on TV images
• CPC Class Codes: H04N 19/137 Motion inside a coding unit, e.g. average field, frame or block difference
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/87 involving scene cut or scene change detection in combination with video compression

Abstract

A method and system for sensing disappearance of the previously detected logos in noisy digital video signals. When a video program starts, a short period of time is allowed to initially detect the existence and the location of the logo within the incoming video frames. Once the logo is detected and a logo map is generated, logo tracking senses whether the logo remains in, or disappears from, the video frames. If the logo moves or (partially) changes its intensity, color, pattern, etc., it will be sensed as having disappeared altogether.

Claims

1. A method of detecting fade in an area of each of a sequence of frames comprising pixels that represent digital video images, comprising:

calculating a stochastic measure σ

_d of the area based on a standard deviation of pixel value differences in at least two frames among the sequence of frames; and

detecting the fade in said area by calculating a difference between the stochastic measure σ

_d and a given noise level, such that if the difference is greater than a threshold value then fade is detected.

2. The method of claim 1, wherein the area comprises a logo area, and wherein calculating the stochastic measurement σ

_d further comprises;

(a) for each pixel in the logo area in a current frame;

(i) determining a difference between a value of a current pixel and that of a corresponding pixel in a reference frame,

(ii) subtracting a mean value of the given noise level from the said difference and squaring the obtained value; and

(iii) generating a sum of the squared differences with noise mean subtracted for each of a plurality of pixels within the logo area;

(b) generating an average of the sums, and determining the stochastic measure σ

_d based on said average.

3. The method of claim 2, wherein generating said sum is according:

$\begin{array}{c}{\mathrm{\sigma <br><br>}}_d^2<br><br>\left(i\right)=<br><br>\underset{j=1}{\overset{i}{\sum <br><br>}}<br><br>\left\{{<br><br>d^{\mathrm{\prime <br><br>}}<br><br>\left(j\right)<br><br>}^2|f<br><br>\left(j\right)=1\right\}\\ =<br><br>\underset{j=1}{\overset{i}{\sum <br><br>}}<br><br>\left\{{<br><br>d<br><br>\left(j\right)-{\mathrm{\mu <br><br>}}_s<br><br>}^2|f<br><br>\left(j\right)=1\right\}\end{array}$ $\text{such <br><br> <br><br>that;<br><br>}$ $d<br><br>\left(j\right)=p_c<br><br>\left(j\right)-p_r<br><br>\left(j\right)$

wherein;

i is the number of pixels that have been examined in the current frame,

P_c(J) denotes the pixel value in the current frame,

P_r(i) denotes the corresponding pixel value in the reference frame,

d(j) denotes the difference between the value of the current pixel and that of a corresponding pixel in the reference frame,

f(j)=1 denotes the pixel being a logo pixel, and

μ

_s denotes the mean of the noise signal.

4. The method of claim 3, wherein calculating the stochastic measure σ

_d is according to;

${\mathrm{\sigma <br><br>}}_d=\sqrt{{\mathrm{\sigma <br><br>}}_d^2},\mathrm{and}$ $\begin{array}{c}{\mathrm{\sigma <br><br>}}_d^2=<br><br>\frac{1}{M}<br><br>\underset{j=1}{\overset{N}{\sum <br><br>}}<br><br>\left\{{<br><br>d^{\mathrm{\prime <br><br>}}<br><br>\left(j\right)<br><br>}^2|f<br><br>\left(j\right)=1\right\}\\ =<br><br>\frac{1}{M}<br><br>\underset{j=1}{\overset{N}{\sum <br><br>}}<br><br>\left\{{<br><br>d<br><br>\left(j\right)-{\mathrm{\mu <br><br>}}_s<br><br>}^2|f<br><br>\left(j\right)=1\right\}\end{array}$

wherein;

M denotes the total number of pixels in said logo area, and

N denotes the total number of pixels in the current frame.

5. The method of claim 2, wherein:

said area includes a sub-image; and

wherein the method further comprises;

initially detecting the area including the sub-image, and

generating a map including a pixel pattern corresponding to the current frame, wherein each pixel in the pixel pattern indicates whether the corresponding pixel in the current frame is in the sub-image.

6. The method of claim 5, further comprising:

using the map to select pixels of the current frame that are in the sub-image for;

determining the difference between the value of the current pixel and that of a corresponding pixel in a reference frame, subtracting the noise mean from the difference and squaring the obtained value; and

generating a sum of the squared values for each of a plurality of pixels within said logo area.

7. The method of claim 1, wherein the noise level is obtained by determining a standard deviation, σ

_s, of noise samples s(i) from the video images, wherein i is the index of a digital noise sample.

8. The method of claim 7, wherein σ

_s, within said area, is expressed as;

${\mathrm{\sigma <br><br>}}_s=\sqrt{\frac{\underset{i=1}{\overset{M}{\sum <br><br>}}<br><br>{<br><br>s<br><br>\left(i\right)-{\mathrm{\mu <br><br>}}_s<br><br>}^2}{M}},\mathrm{when}<br><br><br><br>M-><br><br>\mathrm{\infty <br><br>}$

wherein the noise samples comprise a natural video noise modeled by a random process, the noise samples having a mean value of;

${\mathrm{\mu <br><br>}}_s=\frac{\underset{i=1}{\overset{M}{\sum <br><br>}}<br><br>s<br><br>\left(i\right)}{M},\mathrm{when}<br><br><br><br>M-><br><br>\mathrm{\infty <br><br>}$

wherein M is the total number of noise samples.

25. The method of claim 1,further comprising restarting an initial logo detection and tracking process after detecting the gradual fade and after a short period of time to allow the logo to gradually fade off until the logo disappears.

27. The method of claim 1, wherein the fade is a gradual fade.

9. A method of detecting fade of a logo in a sequence of digital video frames comprising pixels, comprising:

(a) for each pixel in a logo area of a current frame;

(i) determining a difference between a value of a current pixel and that of a corresponding pixel in a reference frame;

(ii) subtracting a mean of a given noise from the said difference and squaring the obtained value; and

(iii) generating a sum of the squared differences with the noise mean subtracted for each of a plurality of pixels within the logo area;

(b) generating an average of the sums;

(c) comparing said average with a level of the given noise characterized by a standard deviation, such that if a difference between the average and the noise standard deviation is greater than a threshold value then fade is detected.

10. The method of claim 9, further comprising:

detecting the logo in the current frame, and

generating a map including a pixel pattern corresponding to the current frame, wherein each pixel in the pixel pattern indicates whether the corresponding pixel in the current frame is in the logo.

11. The method of claim 10, further comprising:

using the map to select pixels of the current frame in the logo area for;

determining the difference between the value of the current pixel and that of a corresponding pixel in a reference frame;

subtracting the noise mean from said difference, and squaring the obtained value; and

generating a sum of said squared values for each of the selected pixels.

12. The method of claim 9, wherein the standard deviation characterizing the given noise level is a standard deviation of noise samples from the video frames.

13. The method of claim 12, wherein the given noise level is determined from a standard deviation of noise samples within the logo area.

26. The method of claim 10, wherein the map comprises pixel values of 1'"'"'s and 0'"'"'s, with the 1'"'"'s corresponding to the detected logo area and the 0'"'"'s corresponding to background video content.

14. A fade detection system for detecting fade in a logo in each of a sequence of video frames, comprising:

a detector for detecting the logo in one of the video frames;

a logo map generator for generating a map including a pixel pattern corresponding to the video frame, wherein each pixel in the pixel pattern indicates whether the corresponding pixel in the video frame is in the logo; and

a logo tracker that detects logo fade by;

using the map to select pixels in the logo from the video frame to determine a difference between a value of the current pixel and a value of a corresponding pixel in a reference frame,

subtracting a given noise mean from the said difference and squaring the obtained value, and

generate a sum of the squared values for each of a plurality of pixels within the logo area;

generating an average of the sums; and

comparing said average with a given noise level, such that if a difference between the said average and the given noise level is greater than the threshold value then fade is detected.

15. The system of claim 14, wherein the given noise level is determined by a standard deviation of noise samples from the video frames.

16. The system of claim 15, wherein the given noise level is determined by a standard deviation of noise samples from pixels in the logo.

17. A fade detection system for detecting fade in a logo in each of a sequence of frames, comprising:

a detector for detecting the logo in one of the frames;

a logo tracker that calculates a stochastic measure σ

_d of said logo based on a standard deviation of differences in pixel values in at least two of said frames, and detects fade in said logo by comparing σ

_d with a given noise level such that if the difference between σ

_d and the given noise level is greater than a threshold value then fade is detected.

18. The system of claim 17, wherein the tracker calculates the stochastic measurement σ

_d by;

(a) for each pixel in a logo area of a current frame;

(i) determining a difference between a value of a current pixel and that of a corresponding pixel in a reference frame;

(ii) subtracting a mean of the given noise level from the said difference, and squaring the obtained value; and

(ii) generating a sum of the squared values for each of a plurality of pixels in the logo area;

(b) generating an average of the sums, and determining the σ

_d based on said average.

19. The system of claim 18, wherein the tracker generates said sum of the squared values, with noise mean subtracted, according to:

$\begin{array}{c}{\mathrm{\sigma <br><br>}}_d^2<br><br>\left(i\right)=<br><br>\underset{j=1}{\overset{i}{\sum <br><br>}}<br><br>\left\{{<br><br>d^{\mathrm{\prime <br><br>}}<br><br>\left(j\right)<br><br>}^2|f<br><br>\left(j\right)=1\right\}\\ =<br><br>\underset{j=1}{\overset{i}{\sum <br><br>}}<br><br>\left\{{<br><br>d<br><br>\left(j\right)-{\mathrm{\mu <br><br>}}_s<br><br>}^2|f<br><br>\left(j\right)=1\right\}\end{array}$ $\text{such <br><br> <br><br>that;<br><br>}$ $d<br><br>\left(j\right)=p_c<br><br>\left(j\right)-p_r<br><br>\left(j\right)$

wherein;

P_c(J) denotes the pixel value in the current frame,

P_r(i) denotes the corresponding pixel value in the reference frame,

d(j) denotes the difference between the value of the current pixel and that of a corresponding pixel in the reference frame,

f(j)=1 denotes that the pixel is a logo pixel, and

μ

_s denotes the mean of the given noise level.

20. The system of claim 19, wherein the tracker determines the σ

_d according to;

${\mathrm{\sigma <br><br>}}_d=\sqrt{{\mathrm{\sigma <br><br>}}_d^2},\mathrm{and}$ $\begin{array}{c}{\mathrm{\sigma <br><br>}}_d^2=<br><br>\frac{1}{M}<br><br>\underset{j=1}{\overset{N}{\sum <br><br>}}<br><br>\{{<br><br>d^{\mathrm{\prime <br><br>}}<br><br>\left(j\right)<br><br>}^2<br><br><br><br>f<br><br>\left(i\right)=1\}\\ =<br><br>\frac{1}{M}<br><br>\underset{j=1}{\overset{N}{\sum <br><br>}}<br><br>\{{<br><br>d<br><br>\left(j\right)-{\mathrm{\mu <br><br>}}_s<br><br>}^2<br><br><br><br>f<br><br>\left(i\right)=1\}\end{array}$

wherein;

M denotes the total number of pixels in said log area, and

N denotes the total number of pixels in the current frame.

21. The system of claim 18, further comprising a logo map generator that generates a map including a pixel pattern corresponding to the current frame, wherein each pixel in the pixel pattern indicates whether the corresponding pixel in the current frame is in the logo.

22. The system of claim 21, wherein the tracker uses the map to select pixels of the current video frame that are in a sub-image for:

determining the difference between the value of the current pixel and that of a corresponding pixel in a reference frame, subtracting the noise mean from the said difference and squaring the obtained value; and

generating a sum of the squared values for each of a plurality of pixels within the logo area.

23. The system of claim 17, wherein the given noise level is determined by a standard deviation, σ

_sof noise samples s(i) from the frames.

24. The system of claim 23, wherein σ

_swithin said area, is expressed as;

${\mathrm{\sigma <br><br>}}_s=\sqrt{\frac{\underset{i=1}{\overset{M}{\sum <br><br>}}<br><br>{<br><br>s<br><br>\left(i\right)-{\mathrm{\mu <br><br>}}_s<br><br>}^2}{M}},\mathrm{when}<br><br><br><br>M\to <br><br>\mathrm{\infty <br><br>}$

wherein a natural video noise signal is modeled by a random process having a mean;

${\mathrm{\mu <br><br>}}_s=\frac{\underset{i=1}{\overset{M}{\sum <br><br>}}<br><br>s<br><br>\left(i\right)}{M},\mathrm{when}<br><br><br><br>M\to <br><br>\mathrm{\infty <br><br>}$

wherein M is the total number of noise samples.