Apparatus and method of detecting advertisement from moving-picture and computer-readable recording medium storing computer program to perform the method

• US 20060245724A1
• Filed: 04/20/2006
• Published: 11/02/2006
• Est. Priority Date: 04/29/2005
• Status: Abandoned Application

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Moon, Yongsu, Kim, Jiyeun, Eom, Kiwan, Hwang, Doosun
• Application Number:
  US11/407,037
• Publication Number:
  US 20060245724A1
• Time in Patent Office: Days
• US Class Current: 386/241
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/812 involving advertisement data advertising per se G06Q30/02
• CPC Class Codes: H04N 21/8456 by decomposing the content in the time domain, e.g. in time segments
• CPC Class Codes: H04N 5/76 Television signal recording
• CPC Class Codes: H04N 7/163 by receiver means only

Abstract

A method of detecting an advertisement in a moving-picture, and an apparatus to perform the method, the method including detecting a component of a visual event from a visual component of the moving-picture, combining or dividing shots based on the component of the visual event, and determining a result obtained by the combination or division of shots as a segment; and detecting an advertisement candidate segment using a rate of shots of the segment; wherein the visual event denotes an effect included in a scene conversion in the moving-picture, the advertisement candidate segment denotes a segment to be a candidate of an advertisement segment, and the advertisement segment denotes a segment having an advertisement as its content.

Claims

1. An apparatus used to detect an advertisement in a moving-picture, the apparatus comprising:

a segment generator to detect a component of a visual event from a visual component of the moving-picture, to combine or divide shots based on the component of the visual event, and to output a result obtained by the combination or division of shots as a segment; and

an advertisement candidate segment detector to detect an advertisement candidate segment using a rate of shots of the segment;

wherein the visual event denotes an effect included in a scene conversion in the moving-picture, the advertisement candidate segment denotes a segment to be a candidate of an advertisement segment, and the advertisement segment denotes a segment having an advertisement as its content.

2. The apparatus of claim 1, wherein the segment generator comprises:

a visual event detector to detect the component of the visual event from the visual component;

a scene conversion detector to detect the scene conversion from the visual component and to generate time and color information of a shot which is a same scene section using a result obtained by the scene conversion detection; and

a visual shot combiner/divider to analyze similarity of the shots using the color information of the shots received from the scene conversion detector, and to combine or divide the shots using the analyzed similarity and the component of the visual event.

3. The apparatus of claim 2, wherein the visual event detector detects a single color frame in a fade effect from the visual component, and outputs the detected single color frame as a component of the visual event,

wherein the visual event is the fade effect.

4. The apparatus of claim 2, wherein the visual event is one of a fade effect, a dissolve effect, and a wipe effect.

5. The apparatus of claim 2, wherein the visual shot combiner/divider comprises;

a buffer to store the color information of the shots received from the scene conversion detector;

a similarity calculator to read color information of a first integral number pertaining to a search window among the color information stored in the buffer, and to calculate color similarity of the shots using the read color information; and

a combiner to compare the color similarity and a similarity threshold, and to combine the compared two shots in response to the result obtained by the comparison of the two shots.

6. The apparatus of claim 5, wherein the visual shot combiner/divider further comprises a divider to divide a result obtained by the combination of the two shots in the combiner based on the component of the visual event in response to the result obtained by the combination including the component of the visual event.

7. The apparatus of claim 5, wherein the similarity calculator calculates the color similarity as

$\mathrm{Sim}$

( H 1 , H 2 ) = ∑

n = 1 N ⁢

min ⁡

[ H 1 ⁡

( n ) , H 2 ⁡

( n ) ]

wherein Sim (H1, H2) denotes the color similarity of the two shots, H1(n) and H2(n) denote color histograms of the two shots, respectively, N denotes a histogram level, and min(x,y) denotes a minimum value of x and y.

8. The apparatus of claim 5, wherein the first integral number, which is the size of the search window, is determined according to EPG information.

9. The apparatus of claim 2, wherein the advertisement candidate segment detector comprises:

a rate calculator to calculate the rate of shots in the segment input from the segment generator using the scene conversion detected in the scene conversion detector;

a rate comparator to compare the shot rate and a rate threshold; and

an advertisement candidate segment output unit to output the segment received in the rate calculator as the advertisement candidate segment in response to the result obtained by the comparison in the rate comparator.

10. The apparatus of claim 9, wherein the rate calculator calculates the shot rate as

$S$

 

⁢

C ⁢

 

⁢

R = S N ⁢

#

wherein SCR (Shots Change Rate within the segment shot) denotes the shot rate, S denotes a number of shots included in the segment generated in the segment generator, and N# denotes a number of frames included in the segment generated in the segment generator.

11. The apparatus of claim 9, wherein the advertisement candidate segment output unit combines or extends the advertisement candidate segments.

12. The apparatus of claim 11, wherein the advertisement candidate segment output unit combines the successive advertisement candidate segments.

13. The apparatus of claim 11, wherein the advertisement candidate segment output unit regards the advertisement non-candidate segment as an advertisement candidate segment, and extends the region of the advertisement candidate segment, in response to an advertisement non candidate segment being included in the advertisement candidate segments;

wherein the advertisement non-candidate segment indicates a segment which is not a candidate of the advertisement segment.

14. The apparatus of claim 1, further comprising:

an acoustic shot characteristic extractor to extract acoustic shot characteristics using the component of an acoustic event detected from an acoustic event of the moving-picture and the segment generated in the segment generator; and

an advertisement segment determiner to determine the advertisement candidate segment to be the advertisement segment using the extracted acoustic shot characteristics;

wherein the acoustic event comprises a sound used to identify the acoustic component.

15. The apparatus of claim 14, wherein the acoustic shot characteristic extractor comprises:

an audio characteristic value generator to extract audio features from the acoustic component by frames, and to output an average and a standard deviation of the audio features of a second integer number of frames as audio characterizing values;

an acoustic event detector to detect the component of the acoustic event using the audio characteristic values; and

a characteristic extractor to extract the acoustic shot characteristics using the detected component of the acoustic event and the segment generated in the segment generator.

16. The apparatus of claim 15, wherein the audio characteristic value generator comprises:

a frame unit divider to divide the acoustic component of the moving-picture into frames of a predetermined time;

a feature extractor to extract each audio feature of the divided frames; and

an average/standard deviation calculator to calculate the average and the standard deviation of the audio features of the second integer number of frames in the feature extractor, and to output the calculated average and standard deviation as the audio characteristic values.

17. The apparatus of claim 15, wherein the audio features comprise MFCC (Mel-Frequency Cepstral Coefficient), Spectral Flux, Centroid, Rolloff, ZCR, Energy, or Picth information, or a combination thereof.

18. The apparatus of claim 15, wherein the component of the acoustic event comprises music, voice, surrounding noise, mute, or a combination thereof.

19. The apparatus of claim 15, wherein the feature extractor outputs at least one of a rate of the component of the acoustic event, a portion of music among components of the acoustic event, and maximum time duration of a sequence comprising components of the same acoustic event as the acoustic shot characteristics in units of the segment generated in the segment generator.

20. The apparatus of claim 19, wherein the rate of the component of the acoustic event, the portion of music among components of the acoustic event, and maximum time duration of a sequence comprising components of the same acoustic event are calculated as

$\mathrm{ACCR}=\frac{\sum _{}^{}}{}$

j = 2 J ⁢

H ⁡

[ C ⁡

( j ) , C ⁡

( j - 1 ) ] J

wherein ACCR (Audio Class Change Rate within the segment shot) denotes the rate of the component of the acoustic event, J denotes a number of audio clips included in the segment generated in the segment generator, the clip is a minimum unit classified as an acoustic component, and C(j) denotes a type of components of the acoustic event of a j^th audio clip, in which if C(j)≠

C(j−

1), H[C(j),C(j−

1)] is ‘

1’

, and if C(j)=C(j−

1), H[C(j),C(j−

1)] is ‘

0’

, then

$\mathrm{MCR}=\frac{\underset{j=1}{\overset{J}{\sum <br><br>}}<br><br>\mathrm{SM}<br><br>\left[C<br><br>\left(j\right),\mathrm{\text{'}"\text{'}"\text{'}\text{'}"\text{'}"\text{'}}<br><br>\mathrm{Music}<br><br>\mathrm{\text{'}"\text{'}"\text{'}\text{'}"\text{'}"\text{'}}\right]}{J}$

wherein MCR (Music Class Ratio within the segment shot) denotes the portion of music among components of the acoustic event, and M denotes the number of sequences comprising components of the same acoustic event included in the segment generated in the segment generator, in which if C(j)=“

Music”

, SM[C(j), “

Music”

] is ‘

1’

, and if C(j)≠

“

Music”

, SM[C(j), “

Music”

] is ‘

0’

, then

$\mathrm{MDS}=\frac{\underset{1\le <br><br>m\le <br><br>M}{\max }<br><br>\left[d_s<br><br>\left(m\right)\right]}{J}$

wherein MDS (Max-Duration of the Sequence with same audio classes within the segment shot) denotes the maximum time duration, and ds(m) denotes the number of audio clips of an m^th sequence.

21. The apparatus of claim 14, wherein the advertisement segment determiner comprises:

a threshold comparator to compare the extracted acoustic shot characteristics and characteristic thresholds; and

an advertisement section determiner to determine the advertisement candidate segment to be the advertisement segment in response to the result obtained by the comparison in the threshold comparator, and to output a beginning and end of the advertisement segment as a beginning and end of the advertisement.

22. The apparatus of claim 14, wherein the advertisement segment determiner comprises:

a threshold comparator to compare the extracted acoustic shot characteristics and characteristic thresholds;

a subtitle checking unit to check whether the advertisement candidate segment includes a subtitle in response to the result obtained by the comparison; and

an advertisement section determiner to determine the advertisement candidate segment to be the advertisement segment in response to the result obtained by the comparison in the subtitle checking unit, and to determine and output a beginning and end of the advertisement segment as a beginning and end of the advertisement;

wherein the advertisement includes the subtitle.

23. The apparatus of claim 1, wherein a result obtained by removing the advertisement candidate segment from segments generated in the segment generator is used as a result obtained by summarizing the moving-picture.

24. The apparatus of claim 14, wherein a result obtained by removing the advertisement segment determined in the advertisement segment determiner from the segments generated in the segment generator is used as the result obtained by summarizing the moving-picture.

25. The apparatus of claim 23, wherein meta data of the result obtained by summarizing the moving-picture is generated, and the generated meta data is stored along with the result obtained by summarizing the moving-picture.

26. The apparatus of claim 1, wherein the visual component of the moving-picture includes both the visual component and EPG information included in a television broadcasting signal.

27. The apparatus of claim 14, wherein the acoustic component of the moving-picture includes both the acoustic component and EPG information included in the television broadcasting signal.

28. A method of detecting an advertisement in a moving-picture, the method comprising:

detecting a component of a visual event from a visual component of the moving-picture, combining or dividing shots based on the component of the visual event, and determining a result obtained by the combination or division of shots as a segment; and

detecting an advertisement candidate segment using a rate of shots of the segment;

wherein the visual event denotes an effect included in a scene conversion in the moving-picture, the advertisement candidate segment denotes a segment to be a candidate of an advertisement segment, and the advertisement segment denotes a segment having an advertisement as its content.

29. The method of claim 28, wherein the determining of the result comprises:

detecting the component of the visual event from the visual component;

detecting the scene conversion from the visual component and generating time and color information of a shot which is a same scene section using a result obtained by the scene conversion detection; and

analyzing similarity of the shots using the color information of the shots, and combining or dividing the shots using the analyzed similarity and the component of the visual event.

30. The method of claim 29, wherein the detecting of the advertisement candidate segment comprises:

calculating a rate of the shot in the determined segment using the detected scene conversion;

determining whether the rate of the shot is higher than a threshold; and

determining the segment used to calculate the rate of the shot to be the advertisement candidate segment in response to the rate of the shot being higher than the threshold.

31. The method of claim 28, further comprising:

extracting acoustic shot characteristics using the component of an acoustic event detected from an acoustic event of the moving-picture and the segment; and

determining the advertisement candidate segment to be the advertisement segment using the extracted acoustic shot characteristics;

wherein the acoustic event is a type of sound used to identify the acoustic component.

32. The method of claim 31, wherein the extracting of the acoustic shot characteristics comprises:

extracting audio features from the acoustic component by frames, and outputting an average and a standard deviation of the audio features of a second integer number of frames as audio characterizing values;

detecting the component of the acoustic event using the audio characteristic values; and

extracting the acoustic shot characteristics using the detected component of the acoustic event and the segment.

33. The method of claim 31, wherein the determining of the advertisement segment determiner comprises:

determining whether the extracted acoustic shot characteristics are larger than characteristic thresholds; and

determining the advertisement candidate segment to be the advertisement segment, and outputting a beginning and an end of the advertisement segment as a beginning and an end of the advertisement, in response to the extracted acoustic shot characteristics being larger than the characteristic thresholds.

34. The method of claim 31, wherein the advertisement segment determiner comprises:

determining whether the extracted acoustic shot characteristics are larger than characteristic thresholds;

determining whether the advertisement candidate segment includes a subtitle in response to the extracted acoustic shot characteristics being larger than the characteristic thresholds; and

determining the advertisement candidate segment to be the advertisement segment, and determining a beginning of the advertisement segment as a beginning of the advertisement, and determining and outputting an end of the detected subtitle as an end of the advertisement, in response to the advertisement candidate segment including the subtitle;

wherein the advertisement includes the subtitle.

35. At least one computer readable medium storing instructions that control at least one processor to perform a method a method of detecting an advertisement in a moving-picture, wherein the method comprises:

detecting a component of a visual event from a visual component of the moving-picture, combining or dividing shots based on the component of the visual event, and determining a result obtained by the combination or division of shots as a segment; and

detecting an advertisement candidate segment using a rate of shots of the segment;

wherein the visual event denotes an effect included in a scene conversion in the moving-picture, the advertisement candidate segment denotes a segment to be a candidate of an advertisement segment, and the advertisement segment denotes a segment having an advertisement as its content.

36. The computer-readable recording medium of claim 35, wherein the method further comprises:

extracting acoustic shot characteristics using the component of an acoustic event detected from an acoustic event of the moving-picture and the segment; and

determining the advertisement candidate segment to be the advertisement segment using the extracted acoustic shot characteristics,

wherein the acoustic event is a sound used to identify the acoustic component.

37. An apparatus used to detect an advertisement in a moving-picture, the apparatus comprising:

a segment generator to detect a visual event from a visual component of the moving-picture, to combine or divide shots according to the visual event, and to output a result of the combination or division as a segment;

wherein an advertisement candidate segment is determined according to a rate of shots of the segment.

38. An apparatus used to detect an advertisement in a moving-picture, the apparatus comprising:

a segment generator to generate a segment of the moving-picture according to visual events detected in the moving picture; and

an advertisement candidate segment detector to detect an advertisement segment according to a rate of shots of the segment;

wherein the shots are combined or divided according to the detected visual events.

39. A method of detecting an advertisement in a moving-picture, the method comprising:

detecting a visual event from a visual component of the moving-picture;

combining or dividing shots according to the visual event; and

outputting a result of the combination or division as a segment;

wherein an advertisement candidate segment is determined according to a rate of shots of the segment.

40. A method of detecting an advertisement in a moving-picture, the method comprising:

generating a segment of the moving-picture according to visual events detected in the moving picture; and

detecting an advertisement segment according to a rate of shots of the segment;

wherein the shots are combined or divided according to the detected visual events.