Method for tracking the disappearance of detected logos within digital video signals

• US 20050078223A1
• Filed: 10/09/2003
• Published: 04/14/2005
• Est. Priority Date: 10/09/2003
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd. (Samsung Group)
• Original Assignee: Samsung Electronics Co. Ltd. (Samsung Group)
• Inventors: Kim, Yeong-Taeg, Liu, Shan
• Granted Patent: US 7,356,084 B2
• Time in Patent Office: Days
• US Class Current: 348/701
• CPC Class Codes: G06V 20/62 Text, e.g. of license plates, overlay texts or captions on TV images
• CPC Class Codes: H04N 19/137 Motion inside a coding unit, e.g. average field, frame or block difference
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/87 involving scene cut or scene change detection in combination with video compression

Abstract

A method and system for sensing disappearance of the previously detected logos in noisy digital video signals. When a video program starts, a short period of time is allowed to initially detect the existence and the location of the logo within the incoming video frames. Once the logo is detected and a logo map is generated, logo tracking senses whether the logo remains in, or disappears from, the video frames. If the logo moves or (partially) changes its intensity, color, pattern, etc., it will be sensed as having disappeared altogether.

Claims

1. A method of detecting fade in an area of each of a sequence of frames comprising pixels that represent digital video images, comprising the steps of:

calculating a stochastic measure σ

_d of said area based on the standard deviation of differences in pixel values in at least two of said frames; and

detecting fade in said area by comparing the stochastic measure σ

_d with a given noise level, such that if the difference between σ

_d and noise level is greater than a threshold value then fade is detected.

2. The method of claim 1 wherein the steps of calculating the stochastic measurement σ

_d further includes the steps of;

(a) for each logo pixel in the current frame;

(i) determining the difference between the value of the current pixel and that of a corresponding pixel in a reference frame,

(ii) subtracting the given noise mean from the said difference and squaring the obtained value; and

(iii) generating a sum of the squared differences with noise mean subtracted for each of a plurality of pixels within the logo area;

(b) generating an average of the sums, and determining the stochastic measure σ

_d based on said average.

3. The method of claim 2, in step (a) generating said sum of the stochastic measure σ

_d is according to the relation;

$\begin{array}{c}{\mathrm{\sigma <br><br>}}_d^2<br><br>\left(i\right)=<br><br>\underset{j=1}{\overset{i}{\sum <br><br>}}<br><br>\left\{{<br><br>d^{\mathrm{\prime <br><br>}}<br><br>\left(j\right)<br><br>}^2|f<br><br>\left(j\right)=1\right\}\\ =<br><br>\underset{j=1}{\overset{i}{\sum <br><br>}}<br><br>\left\{{<br><br>d<br><br>\left(j\right)-{\mathrm{\mu <br><br>}}_s<br><br>}^2|f<br><br>\left(j\right)=1\right\}\end{array}$ $\text{such <br><br> <br><br>that;<br><br>}$ $d<br><br>\left(j\right)=p_c<br><br>\left(j\right)-p_r<br><br>\left(j\right)$

wherein;

i is the number of pixels that have been examined in the current frame,

p_c(j) denotes the pixel value in a current video frame,

p_r(j) denotes the corresponding pixel intensity value in the reference frame,

d(j) denotes the difference between the value of the current pixel and that of a corresponding pixel in the reference frame,

f(j)=1 denotes the pixel is a logo pixel, and

μ

_s denotes the mean of the noise signal.

4. The method of claim 3, wherein determining the stochastic measure σ

_d is according to the relations;

${\mathrm{\sigma <br><br>}}_d=\sqrt{{\mathrm{\sigma <br><br>}}_d^2},\mathrm{and}$ $\begin{array}{c}{\mathrm{\sigma <br><br>}}_d^2=<br><br>\frac{1}{M}<br><br>\underset{j=1}{\overset{N}{\sum <br><br>}}<br><br>\left\{{<br><br>d^{\mathrm{\prime <br><br>}}<br><br>\left(j\right)<br><br>}^2|f<br><br>\left(j\right)=1\right\}\\ =<br><br>\frac{1}{M}<br><br>\underset{j=1}{\overset{N}{\sum <br><br>}}<br><br>\left\{{<br><br>d<br><br>\left(j\right)-{\mathrm{\mu <br><br>}}_s<br><br>}^2|f<br><br>\left(j\right)=1\right\}\end{array}$

wherein;

M denotes the total number of pixels in said logo area, and

N denotes the total number of pixels in the current frame.

5. The method of claim 2 wherein:

said area includes a sub-image;

the method further includes the steps of initially detecting the area including the sub-image, and generating a map including a pixel pattern corresponding to the video frame, wherein each pixel in the pattern indicates whether the corresponding pixel in the video frame is in the sub-image.

6. The method of claim 5, further including the steps of:

using the map to select pixels of the video frame that are in the sub-image for;

determining the difference between the value of the current pixel and that of a corresponding pixel in a reference frame, subtracting the noise mean from the difference and squaring the obtained value; and

generating a sum of the squared values for each of a plurality of pixels within the said logo area.

7. The method of claim 1, wherein the noise level is selected by determining a standard deviation of noise samples s(i) from the video signal, wherein i is the index of a digital noise sample.

8. The method of claim 7, wherein the standard deviation of noise, σ

_s, within said area can be expressed as;

${\mathrm{\sigma <br><br>}}_s=\sqrt{\frac{\underset{i=1}{\overset{M}{\sum <br><br>}}<br><br>{<br><br>s<br><br>\left(i\right)-{\mathrm{\mu <br><br>}}_s<br><br>}^2}{M}},\mathrm{when}<br><br>\text{\hspace{1em}<br><br>}<br><br>M-><br><br>\mathrm{\infty <br><br>}.$

wherein a natural video noise signal is modeled by a random process with mean;

${\mathrm{\mu <br><br>}}_s=\frac{\underset{i=1}{\overset{M}{\sum <br><br>}}<br><br>s<br><br>\left(i\right)}{M},\mathrm{when}<br><br>\text{\hspace{1em}<br><br>}<br><br>M-><br><br>\mathrm{\infty <br><br>}$

wherein M is the total number of noise samples.

9. A method of detecting fade of a logo in a sequence of frames comprising pixels that represent digital video images, comprising the steps of:

(a) for each logo pixel in the current frame;

(i) determining the difference between the value of the current pixel and that of a corresponding pixel in a reference frame;

(ii) subtracting the given noise mean from the said difference and squaring the obtained value; and

(iii) generating a sum of the squared differences with noise mean subtracted for each of a plurality of pixels within the logo area;

(b) generating an average of the sums;

(c) comparing said average to a given noise level represented by STD, such that if the difference between the average and the noise STD is greater than the threshold value then fade is detected.

10. The method of claim 9, further including the initial steps of:

detecting the logo in the video frame, and generating a map including a pixel pattern corresponding to the video frame, wherein each pixel in the pattern indicates whether the corresponding pixel in the video frame is in the logo.

11. The method of claim 10, further comprising the steps of:

using the map to select pixels of the video frame that are in the logo for;

determining the difference between the value of the current pixel and that of a corresponding pixel in a reference frame;

subtracting the noise mean from said difference, and squaring the obtained value; and

generating a sum of said squared values for each of a plurality of pixels within the logo area.

12. The method of claim 9, wherein the noise level is selected by determining a standard deviation of noise samples from the video signal.

13. The method of claim 12, wherein the noise level is selected by determining a standard deviation of noise samples within the logo area.

14. A fade detection system for detecting fade in a logo in each of a sequence of frames, comprising:

a detector for detecting the logo in a video frame;

a logo map generator generating a map including a pixel pattern corresponding to the video frame, wherein each pixel in the pattern indicates whether the corresponding pixel in the video frame is in the logo;

a logo tracker that detects logo fade by;

using the map to select pixels of the video frame that are in the logo to, determine the difference between the value of the current pixel and that of a corresponding pixel in a reference frame, subtracting the given noise mean from the said difference and squaring the obtained value, and generate a sum of the squared values for each of a plurality of pixels within the logo area;

generating an average of the sums; and

comparing said average to a given noise level, such that if the difference between the said average and the noise level (STD) is greater than the threshold value then fade is detected.

15. The system of claim 14, wherein the noise level is selected by determining a standard deviation of noise samples from the video signal.

16. The system of claim 15, wherein the noise level is selected by determining a standard deviation of noise samples from the logo pixels.

17. A fade detection system for detecting fade in a logo in each of a sequence of frames, comprising:

a detector for detecting the logo in a video signal;

a logo tracker that calculates a stochastic measure σ

_d of said logo based on the standard deviation of differences in pixel values in at least two of said frames, and detects fade in said logo by comparing the measure σ

_d with a given noise level (STD), such that if the difference between σ

_d and the noise level (STD) is greater than the threshold value then fade is detected.

18. The system of claim 17 wherein the tracker calculates the stochastic measurement σ

_d by;

(a) for each pixel in the logo area of the current frame;

(i) determining the difference between the value of the current pixel and that of a corresponding pixel in a reference frame;

(ii) subtracting the noise mean from the said difference, and squaring the obtained value; and

(ii) generating a sum of the squared values for each of a plurality of pixels in the logo area;

(b) generating an average of the sums, and determining the stochastic measure σ

_d based on said average.

19. The system of claim 18, wherein the tracker generates said sum of the squared differences, with noise mean subtracted, according to the relation:

$\begin{array}{c}{\mathrm{\sigma <br><br>}}_d^2<br><br>\left(i\right)=<br><br>\underset{j=1}{\overset{i}{\sum <br><br>}}<br><br>\left\{{<br><br>d^{\mathrm{\prime <br><br>}}<br><br>\left(j\right)<br><br>}^2|f<br><br>\left(j\right)=1\right\}\\ =<br><br>\underset{j=1}{\overset{i}{\sum <br><br>}}<br><br>\left\{{<br><br>d<br><br>\left(j\right)-{\mathrm{\mu <br><br>}}_s<br><br>}^2|f<br><br>\left(j\right)=1\right\}\end{array}$ $\text{such <br><br> <br><br>that;<br><br>}$ $d<br><br>\left(j\right)=p_c<br><br>\left(j\right)-p_r<br><br>\left(j\right)$

wherein;

p_c(j) denotes the pixel value in a current video frame,

p_r(j) denotes the corresponding pixel intensity value in the reference frame,

d(j) denotes the absolute difference between the value of the current pixel and that of a corresponding pixel in the reference frame,

f(j)=1 denotes that the pixel is a logo pixel, and

μ

_s denotes the mean of the noise signal.

20. The system of claim 19, wherein the tracker determines the stochastic measure σ

_d according to the relations;

${\mathrm{\sigma <br><br>}}_d=\sqrt{{\mathrm{\sigma <br><br>}}_d^2},\mathrm{and}$ $\begin{array}{c}{\mathrm{\sigma <br><br>}}_d^2=<br><br>\frac{1}{M}<br><br>\underset{j=1}{\overset{N}{\sum <br><br>}}<br><br>\{{<br><br>d^{\mathrm{\prime <br><br>}}<br><br>\left(j\right)<br><br>}^2<br><br><br><br>f<br><br>\left(i\right)=1\}\\ =<br><br>\frac{1}{M}<br><br>\underset{j=1}{\overset{N}{\sum <br><br>}}<br><br>\{{<br><br>d<br><br>\left(j\right)-{\mathrm{\mu <br><br>}}_s<br><br>}^2<br><br><br><br>f<br><br>\left(i\right)=1\}\end{array}$

wherein;

M denotes the total number of logo pixels in said area, and

N denotes the total number of pixels in the current frame.

21. The system of claim 18, further comprising a logo map generator that generates a map including a pixel pattern corresponding to the video frame, wherein each pixel in the pattern indicates whether the corresponding pixel in the video frame is in the logo.

22. The system of claim 21, wherein the tracker uses the logo map to select pixels of the video frame that are in the sub-image for:

determining the difference between the value of the current pixel and that of a corresponding pixel in a reference frame, subtracting the noise mean from the said difference and squaring the obtained value; and

generating a sum of the squared values for each of a plurality of pixels within the logo area.

23. The system of claim 17, wherein the noise level is selected by determining a standard deviation of noise samples s(i) from the video signal.

24. The system of claim 23, wherein the standard deviation of noise, σ

_s, within said area can be expressed as;

${\mathrm{\sigma <br><br>}}_s=\sqrt{\frac{\underset{i=1}{\overset{M}{\sum <br><br>}}<br><br>{<br><br>s<br><br>\left(i\right)-{\mathrm{\mu <br><br>}}_s<br><br>}^2}{M}},\mathrm{when}<br><br>\text{\hspace{1em}<br><br>}<br><br>M\to <br><br>\mathrm{\infty <br><br>}.$

wherein a natural video noise signal is modeled by a random process with mean;

${\mathrm{\mu <br><br>}}_s=\frac{\underset{i=1}{\overset{M}{\sum <br><br>}}<br><br>s<br><br>\left(i\right)}{M},\mathrm{when}<br><br>\text{\hspace{1em}<br><br>}<br><br>M\to <br><br>\mathrm{\infty <br><br>}$

wherein M is the total number of noise samples.