Media Summarization

• US 20140072285A1
• Filed: 09/10/2012
• Published: 03/13/2014
• Est. Priority Date: 09/10/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.), Delta Electronics Incorporated
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Zomet, Asaf, Chechik, Gal, Shynar, Michael
• Granted Patent: US 9,560,332 B2
• Time in Patent Office: Days
• US Class Current: 386/286
• CPC Class Codes: G11B 27/031 Electronic editing of digitised analogue information signals, e.g. audio or video signals
• CPC Class Codes: H04N 21/252 Processing of multiple end-users' preferences to derive collaborative data
• CPC Class Codes: H04N 21/8549 Creating video summaries, e.g. movie trailer retrieval in video databases by using presentations in form of a video summary G06F16/739
• CPC Class Codes: H04N 5/76 Television signal recording
• CPC Class Codes: H04N 5/765 Interface circuits between an apparatus for recording and another apparatus
• CPC Class Codes: H04N 9/8205 involving the multiplexing of an additional signal and the colour video signal
• CPC Class Codes: H04N 9/87 Regeneration of colour television signals H04N9/80 takes precedence

Abstract

Techniques for summarizing media are described. A viewer-interaction analyzer receives a media file containing media, the media file including a plurality of segments. A segment of the media file is scored based on interactions of a set of raters. Viewer metrics on the segment of the media file are measured based on interactions with the segment of the media file by a set of viewers. A set of feature vectors are formed based on the measured viewer interactions, where feature vectors in the set of feature vectors are based on interactions of the set of viewers. A model is trained based on the set of feature vectors and the score assigned to the segment of the media file. The model is applied to segments of the media file to generate an interest rating for segments of the media file. An edited media file is generated based on segments of the media file having interest ratings that meet a criterion. The Abstract is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims.

Claims

1. At least one computing device that facilitates summarize media, the computing device comprising:

a data store configured to store media;

a viewer-interaction analyzer configured to;

receive a file containing media, the media file including a plurality of segments;

score a segment of the media file based on the actions of a set of raters;

measure viewer metrics on the segment of the media file based on interactions with the segment of the media file by a set of viewers;

form a set of feature vectors based on the measured viewer interactions, where feature vectors in the set of feature vectors are based on interactions of the set of viewers;

train a model based on the set of feature vectors and the score assigned to the segment of the media file; and

apply the model to segments of the media file to generate an interest score for segments of the media file; and

a video editor configured to;

generate an edited media file based on segments of the file having interest ratings that meet a criteria.

2. The at least one computing device of claim 1 wherein the viewer-interaction analyzer is further configured to measure viewer metrics selected from a group consisting of viewership repeating a segment, viewership skipping a segment, viewership pausing on a segment, viewership drop-off, viewership full-screen, viewership annotating, viewership sharing, viewership comment posting, viewership chatting, viewership transitioning from a seek action to a play action, viewership transitioning between a full screen and a small screen, and combinations thereof.

3. The at least one computing device of claim 1 further comprising a video editor configured to edit the media file by filtering from the media file those segments of the media file whose interest score falls below a criteria and memorialize the edited media file.

4. The at least one computing device of claim 3 wherein the viewer-interaction analyzer is further configured to memorialize the edited media file by storing the edited media file in the data store as a summarized version of the media file.

5. The at least one computing device of claim 1 wherein the viewer-interaction analyzer is further configured to train the model based on regression analysis.

6. The at least one computing device of claim 1 wherein the viewer-interaction analyzer is further configured to measure interest of the set of viewers in the segment of the media file selected from the group consisting of repeating a segment, skipping a segment, pausing on a segment, and combinations thereof.

7. The at least one computing device of claim 1 wherein the viewer-interaction analyzer is further configured to score a segment of the media file based on the interest of a set of raters and measure viewer metrics on the segment of the media file based on a set of viewers interacting with the segment of the media file, wherein at least some raters are in the second set of viewers.

8. The at least one computing device of claim 1 wherein the viewer-interaction analyzer is further configured to summarize video.

9. A method implemented by one or more computing devices configured to summarize media, the method comprising:

receiving a file containing media, the media file including a plurality of segments;

scoring a segment of the media file based on interactions of a first set of raters;

measuring viewer metrics on the segment of the media file based on interactions with the segment of the media file by a set of viewers;

forming a set of feature vectors based on the measured viewer interactions, where feature vectors in the set of feature vectors are based on interactions of the set of viewers;

training a model based on the set of feature vectors and the score assigned to the segment of the media file;

applying the model to segments of the media file to generate an interest rating for segments of the media file;

generating an edited media file based on segments of the media file having interest ratings that meet a criteria; and

storing the edited media file in a data store as a summarized version of the media file.

10. The method of claim 9 further comprising editing the media file by filtering from the media file those segments of the media file whose interest score falls below a criteria.

11. The method of claim 9 further comprising training the model based on regression analysis.

12. The method of claim 9 further comprising scoring a segment of the media file based on measuring interest of the at least first set of viewers in the segment of the media file, then estimating the interest of viewers using the model and the measured interactions.

13. One or more computer-readable media storing processor-executable instructions that, when executed, cause one or more processors to perform operations that recommend related media, the operations comprising:

receiving a file containing media, the media file including a plurality of segments;

scoring a segment of the media file based on interactions of a first set of raters;

measuring viewer metrics on the segment of the media file based on interactions with the segment of the media file by a set of viewers;

forming a set of feature vectors based on the measured viewer interactions, where feature vectors in the set of feature vectors are based on interactions of the set of viewers;

training a model based on the set of feature vectors and the score assigned to the segment of the media file;

applying the model to segments of the media file to generate an interest rating for segments of the media file;

generating an edited media file based on segments of the media file having interest ratings that meet a criteria; and

storing the edited media file in a data store as a summarized version of the media file.

14. The one or more computer-readable media storing processor-executable instructions of claim 13 further comprising editing the media file by filtering from the media file those segments of the media file whose interest score falls below a threshold.

15. The one or more computer-readable media storing processor-executable instructions of claim 13 further comprising training the model based on regression analysis.

16. The one or more computer-readable media storing processor-executable instructions of claim 13 further comprising scoring a segment of the media file based on measuring interest of the raters in the segment of the media file, then estimating the interest of viewers using the model and the measured interactions.

17. The one or more computer-readable media storing processor-executable instructions of claim 13 further comprising scoring a segment of the media file based on the interest of the set of raters and measuring viewer metrics on the segment of the media file based on a set of viewers interacting with the segment of the media file, wherein at least some raters are in the set of viewers.