ADVERTISEMENT INSERTION POINTS DETECTION FOR ONLINE VIDEO ADVERTISING

• US 20090079871A1
• Filed: 09/20/2007
• Published: 03/26/2009
• Est. Priority Date: 09/20/2007
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Mei, Tao, Hua, Xian-Sheng, Li, Shipeng, Yang, Linjun
• Granted Patent: US 8,654,255 B2
• Time in Patent Office: Days
• US Class Current: 348/584
• CPC Class Codes: G11B 27/036 Insert-editing
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: H04N 21/2365 Multiplexing of several video streams
• CPC Class Codes: H04N 21/4347 Demultiplexing of several video streams
• CPC Class Codes: H04N 21/812 involving advertisement data advertising per se G06Q30/02
• CPC Class Codes: H04N 21/8456 by decomposing the content in the time domain, e.g. in time segments
• CPC Class Codes: H04N 21/854 Content authoring

Abstract

Systems and methods for determining insertion points in a first video stream are described. The insertions points being configured for inserting at least one second video into the first video. In accordance with one embodiment, a method for determining the insertion points includes parsing the first video into a plurality of shots. The plurality of shots includes one or more shot boundaries. The method then determines one or more insertion points by balancing a discontinuity metric and an attractiveness metric of each shot boundary.

Claims

1. A method, comprising:

parsing a first video into a plurality of shots, the plurality of shots including one or more shot boundaries; and

determining one or more insertion points for inserting at least one second video into the first video, wherein the determination includes balancing a discontinuity and an attractiveness of each of the shot boundaries.

2. The method of claim 1, wherein determining one or more insertion points includes:

computing a degree of discontinuity for each of the one or more shot boundaries;

computing a degree of attractiveness for each of the one or more shot boundaries;

determining one or more insertion points based on the degree of discontinuity and the degree of attractiveness of each shot boundary; and

inserting the at least one second video at each of the determined insert points to form an integrated video stream.

3. The method of claim 1, further comprising providing the integrated video stream for playback, and assessing the effectiveness of the insertion points based on viewer feedback to a played integrated video stream.

4. The method of claim 1, wherein parsing a first video includes parsing the first video into a plurality of shots based on one of visual details of the first video or audio stream of the first video.

5. The method of claim 4, wherein parsing the first video into a plurality of shots based on the visual details of the first video includes using one of a pair-wise comparison method, a likelihood ratio method, an intensity level histogram method, or a twin-comparison method, and wherein parsing the first video into a plurality of shots based on the audio stream of the first video includes using a plurality of kernel support vector machines.

6. The method of claim 2, wherein determining one or more insertion points includes finding peaks in a linear combination of the degrees of discontinuity and the degrees of attractiveness of the one or more shot boundaries.

7. The method of claim 2, wherein determining one or more insertion points further includes linearly combining the degrees of discontinuity of one or more visual shot boundaries and one or more the audio shot boundaries.

8. The method of claim 2, wherein computing a degree of discontinuity for each of the one or more shot boundaries includes computing at least one of a degree of content discontinuity or a degree of semantic discontinuity for each shot boundary.

9. The method of claim 8, wherein computing a degree of content discontinuity for each shot boundary includes:

using the best first merge method to merge the one or more pairs of adjacent shots, wherein the each pair of adjacent shots includes a shot boundary;

assigning a merging order to each shot boundary, the merging order being assigned based on a chronological order in which a corresponding pair of adjacent shots are merged; and

calculating a content discontinuity value for each shot boundary based on the corresponding merging order.

10. The method of claim 8, wherein computing a degree of semantic discontinuity for each of the one or more shot boundaries includes:

using one or more first concept detectors to determine a first confidence score between a first shot and at least one corresponding first attribute;

using one or more second concept detectors to determine a second confidence score between a second shot and at least one corresponding second attribute;

composing the at least one first confidence score into a first vector,

composing the at least one second confidence score into a second vector; and

calculating a semantic discontinuity value for a shot boundary between the first and second shots based on the difference between the first vector and the second vector.

11. The method of claim 10, wherein using one or more first concept detectors and one or more second concept detectors includes using concept detectors that are based on Support Vector Machines (SVMs).

12. The method of claim 2, wherein computing a degree of attractiveness includes computing a degree of attractiveness for each of the plurality of shots using at least one model from a group of visual attention models, the group comprising a motion attention model, a static attention model, a semantic attention model, and guided attention model.

13. The method of claim 12, wherein computing a degree of attractiveness further includes computing a degree of attractiveness for each of the plurality of shot using at least one model from a group of audio attention model models, the group comprising an audio saliency model, a speech attention model, and a music attention model.

14. The method of claim 13, wherein computing a degree of attractiveness for each of the one or more shot boundaries further includes:

obtaining one or more visual attractiveness values for at least one shot form the corresponding group of visual attention models;

obtaining one or more audio attractiveness values for the at least one shot form the corresponding group of audio attention models; and

combining the one or more visual attractiveness values and the one or more audio attractive values using one of a linear fusion scheme or a nonlinear fusion scheme to obtain the degree of attractiveness for the at least one shot.

15. The method of claim 14, wherein computing a degree of attractiveness for each of the one or more shot boundaries further includes:

acquiring a first degree of attractiveness for a first shot;

acquiring a second degree of attractiveness for a second shot, wherein the second shot is adjacent the first shot;

computing a degree of overall attractiveness, A(p_i), by computing the linear combination of attractiveness degrees from the first and second shots according to;

A(p_i)=λ

×

A(s_i)+(1−

λ

)×

A(s_i+1)

wherein A(s_i) represents the degree of attractive for the first shot, and A(s_i) represents the degree of attractiveness for a second shot, and λ

is a number between 0 and 1.

16. The method of claim 2, wherein inserting the at least one second video includes:

acquiring at least one degree of discontinuity for the one or more shot boundaries;

acquiring at least one degree of attractiveness for the one or more shot boundaries;

providing a linear combination curve of the at least one degree of discontinuity and the at least one degree of attractiveness; and

determining the location of the at least one insertion points based on one or more peaks on the linear combination curve.

17. A computer readable medium having computer-executable instructions that, when executed, perform acts comprising:

parsing a first video into a plurality of shots, the plurality of shots includes one or more shot boundaries;

computing a degree of overall discontinuity for each of the one or more shot boundaries;

computing a degree of attractiveness for each of the one or more shot boundaries;

determining one or more insertion points based on the degree of discontinuity and the degree of attractiveness of each shot boundary, wherein the one or more insertion points are configured for inserting at least one second video into the first video; and

inserting the at least one second video at each of the determined insert points to form an integrated video stream.

18. The computer readable medium of claim 17, further comprising providing the integrated video stream for playback, and assessing the effectiveness of the insertion points based on viewer feedback to a played integrated video stream.

19. The computer readable medium of claim 17, wherein parsing a first video includes parsing the first video into a plurality of shots using one of a pair-wise comparison method, a likelihood ratio method, an intensity level histogram method, or a twin-comparison method.

20. The computer readable medium of claim 17, wherein computing a degree of discontinuity for each shot boundary includes:

computing a degree of content discontinuity for a shot boundary;

computing a degree of semantic discontinuity for the shot boundary; and

computing a degree of overall discontinuity based on the average of the degree of content discontinuity and the degree of semantic discontinuity.

21. The computer readable medium of claim 17, wherein computing a degree of discontinuity for each shot boundary includes:

computing a degree of content discontinuity for a shot boundary;

computing a degree of semantic discontinuity for the shot boundary; and

computing a degree of overall discontinuity, D(p_i), by combining the degree of content discontinuity and the degree of semantic discontinuity according to;

D(p_i)=λ

D_c(p_i)+(1−

λ

)D_s(p_i)

wherein D_c(p_i) represents the degree of content discontinuity, D_s(p_i) represents the degree of semantic discontinuity, and λ

is a number between 0 and 1.

22. The computer readable medium of claim 17, wherein inserting the at least one second video includes:

acquiring at least one degree of discontinuity for the one or more shot boundaries;

acquiring at least one degree of attractiveness for the one or more shot boundaries;

providing a linear combination curve of the at least one degree of discontinuity and the at least one degree of attractiveness; and

determining the location of the at least one insertion point based on one or more peaks on the linear combination curve.

23. A system, the system comprising:

one or more processors; and

memory allocated for storing a plurality of computer-executable instructions which are executable by the one or more processors, the computer-executable instructions comprising;

instructions for parsing a first video into a plurality of shots, the plurality of shots includes one or more shot boundaries;

instructions for computing a degree of discontinuity for each of the one or more shot boundaries;

instructions for computing a degree of attractiveness for each of the one or more shot boundaries;

instructions for determining one or more insertion points based on the degree of discontinuity and the degree of attractiveness of each shot boundary, wherein the one or more insertion points are configured for inserting at least one second video into the first video; and

instructions for inserting the at least one second video at each of the determined insert points to form an integrated video stream.

24. The system of claim 23, further comprising instructions for providing the integrated video stream for playback, and assessing the effectiveness of the insertion points based on viewer feedback to a played integrated video stream.