CHARACTER-BASED AUTOMATED MEDIA SUMMARIZATION

• US 20090100454A1
• Filed: 04/23/2007
• Published: 04/16/2009
• Est. Priority Date: 04/25/2006
• Status: Active Grant

• Current Assignee: Nutshell Media, Inc.
• Original Assignee: Nutshell Media, Inc.
• Inventors: Weber, Frank Elmo
• Granted Patent: US 8,392,183 B2
• Time in Patent Office: Days
• US Class Current: 725/9
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence

Abstract

Methods, devices, systems and tools are presented that allow the summarization of text, audio, and audiovisual presentations, such as movies, into less lengthy forms. High-content media files are shortened in a manner that preserves important details, by splitting the files into segments, rating the segments, and reassembling preferred segments into a final abridged piece. Summarization of media can be customized by user selection of criteria, and opens new possibilities for delivering entertainment, news, and information in the form of dense, information-rich content that can be viewed by means of broadcast or cable distribution, “on-demand” distribution, internet and cell phone digital video streaming, or can be downloaded onto an iPod™ and other portable video playback devices.

Claims

1. A method for summarizing an audiovisual presentation, comprising:

a) providing an audiovisual presentation having a first temporal length, the audiovisual presentation comprising;

i) an original time order, and

ii) a plurality of dialogue segments, each dialogue segment comprising a segment component;

b) rating each dialogue segment by determining a significance value for each dialogue segment; and

c) using a significance value threshold to select the dialogue segment if the significance value exceeds the significance value threshold, and delete the dialogue segment if the significance value is below the significance value threshold, thereby forming a summarized version of the audiovisual presentation having a second temporal length.

2. The method of claim 1, wherein the segment component comprises a word, the word having a word significance value, and wherein the significance value of each dialogue segment is determined by a combined rating of word significance values of each segment component in the dialogue segment.

3. The method of claim 2, wherein the word significance values are determined by at least one word characteristic selected from the group consisting of whether the word is a noun or a pronoun, whether the word is a reference to a character, an identity of the character referenced, a frequency of occurrence of the word in the audiovisual presentation, a match of the word to a predetermined database word.

4. The method of claim 3, wherein the word is the reference to a character and the word significance value is determined by a frequency of use of the character'"'"'s name in the audiovisual presentation and a frequency of use of a pronoun referring to the character in the audiovisual presentation.

5. The method of claim 4, wherein the pronoun is a first person pronoun matched to a speaking character, the speaking character determined by a method selected from the group consisting of a voice imprint recognition analysis, a face recognition image analysis, an embedded subtitle analysis, and a manual analysis for the purpose of antecedent identification of the first person pronoun.

6. The method of claim 4, wherein the pronoun is a second person pronoun matched to a non-speaking character when only two characters are present in a scene containing a dialogue segment, wherein the non-speaking character is determined by a method selected from the group consisting of a face recognition image analysis and a manual analysis for the purpose of antecedent identification of the second person pronoun.

7. The method of claim 4, wherein an unmatched pronoun is assigned a pronoun significance value determined by a method selected from the group consisting of averaging the word significance values of all characters referred to, averaging the word significance value of characters referred to within a predetermined number of adjacent dialogue segments, and averaging the word significance value of a predetermined number of most frequently referred to characters.

8. The method of claim 4, wherein the pronoun is identified by comparing the pronoun used in the dialogue segment with a database of pronoun type sets, and wherein the pronoun is matched to an antecedent noun based on proximity of the pronoun to the antecedent noun.

9. The method of claim 4, wherein a relative pronoun is ignored.

10. The method of claim 3 further comprising comparing the segment components with a verb set and an adjective set to determine negativity of the dialogue segment to weight the word significance value based on a predetermined set of values.

11. The method of claim 2, further comprising:

a) analyzing a group of dialogue segments, wherein the group of dialogue segments comprises a first cluster of rated dialogue segments having a first mode, followed by at least one countervailing rated dialogue segment having a countervailing mode opposite that of the first mode, followed by a second cluster of rated dialogue segments having a second mode that is the same as the first mode;

b) determining a percent of composition of the countervailing rated dialogue segments to the total number of dialogue segments in the group of dialogue segments; and

c) converting the countervailing mode to that of the first and second mode if the percent composition is less than a predetermined value.

12. The method of claim 11, wherein the converted countervailing mode is reversed if the absolute value of the significance value deviance of the countervailing rated dialogue segment is greater than the average of the absolute values of the significance value deviances of the two dialogue segments immediately adjacent to the countervailing rated dialogue segment.

13. The method of claim 1, wherein the audiovisual presentation comprises secondary segments selected from the group consisting of a non-dialogue segment, a non-dialogue segment with music segment, a shot segment, and an audience reaction segment, wherein the secondary segments each comprises a secondary significance value to weight the significance value of the dialogue segment.

14. The method of claim 13, wherein the significance value is weighted by the audience reaction, and determining the secondary significance value of the audience reaction further comprises:

a) separating the audience reaction into reaction segments;

b) characterizing the reaction segments in terms of at least one reaction parameter selected from the group consisting of reaction type, reaction length, and reaction intensity; and

c) rating the characterized audience reaction based on the combined rating of the reaction segments for the audience reaction to weight the significance value preceding the audience reaction.

15. The method of claim 13, wherein the weighting of the significance value comprises plotting the significance value of the dialogue segment and at least one secondary significance value as a significance curve overlay on a two-axis graph representing the significance value and the at least one secondary significance value as a function of time in order to weight the significance value for each dialogue segment.

16. The method of claim 1, wherein the significance value is weighted by a perceived presence of stress and the perceived presence of stress is determined by assessing a character communication parameter selected from the group consisting of a volume of speech, a tone of speech, a rate of speech, and a facial expression.

17. The method of claim 1 further comprising smoothing the summarized version of the audiovisual presentation using contour interpolation, comprising:

a) preparing a multiple axis plot of final segment attribute values over time;

b) connecting the plotted segment values creating a graph that resembles a cross-section of mountains and valleys;

c) tracing the plotted line;

d) resisting a predetermined maximum angle deviance of a predetermined input; and

e) effectively removing sharp peaks and crevices of said cross-sections, thereby minimizing content adjacency artifacts.

18. The method of claim 1 further comprising:

a) receiving a playback time request within which to play the summarized audiovisual presentation; and

b) automatically calculating the significance value threshold so that the sunmmarized audiovisual presentation fits within the playback time request.

19. The method of claim 1, wherein the audiovisual presentation is stored on a storage device comprising a center track comprising the dialogue segment and a plurality of peripheral tracks surrounding the center track, the peripheral tracks comprising non-dialogue sound, wherein the dialogue segment rated is from the center track to filter out the non-dialogue sounds to increase the accuracy of analyzing the dialogue segment.

20. The method of claim 1, wherein the audiovisual presentation further comprises an original non-dialogue segment with music, wherein a predetermined first portion of the original non-dialogue segment with music and a predetermined last portion of the original non-dialogue segment with music are selected to include in the summarized version of the audiovisual presentation and a predetermined middle portion of the original non-dialogue segment with music is excluded from the summarized version of the audiovisual presentation, wherein the ratio of the combined length of the included portions to the length of the original non-dialogue segment with music is equal to the ratio of the second temporal length to the first temporal length.

21. The method of claim 1, wherein each dialogue segment is a sentence and the segment component is a word having a word significance value, wherein the significance value of each dialogue segment is determined by a combined rating of the word significance value of each segment component in the sentence.

22. The method of claim 21, wherein the word significance value is determined by comparing the word to at least one database of words selected from the group consisting of a user-defined database, a field of expertise database, and a noun database to determine whether the word segment matches any word in the at least one database, wherein a match increases the word significance value.

23. The method of claim 22, wherein the word is a reference to a character having a noun significance value, wherein the noun significance value is determined by a frequency of references to the character in the audiovisual work.

24. The method of claim 23 further comprising:

a) identifying a pronoun in the sentence by comparing the noun in the sentence to a database of pronoun type sets; and

matching the pronoun to an antecedent noun based on proximity of the pronoun and the antecedent noun in the sentence to increase the noun significance value of the noun.

25. The method of claim 24, wherein the sentence segment significance threshold is determined automatically when a user determines a percentage of truncation of the audiovisual presentation.

26. The method of claim 22, further comprising:

a) analyzing a group of sentences, wherein the group of sentences comprises a first cluster of rated sentences having a first mode, followed by at least one countervailing rated sentence having a countervailing mode opposite that of the first mode, followed by a second cluster of rated sentences having a second mode that is the same as the first mode;

b) determining a percent of composition of the countervailing rated sentence to the total number of sentences in the group of sentences; and

c) converting the countervailing mode to the first and second mode if the percent composition is less than a predetermined value.

27. The method of claim 26, wherein the converted countervailing mode is reversed if the absolute value of the significance value deviance of the countervailing rated sentence is greater than the average of the absolute values of the significance value deviances of the two sentences immediately adjacent to the countervailing rated sentence.

28. The method of claim 21 further comprising smoothing the summarized version of the audiovisual presentation using contour interpolation, comprising:

a) preparing a multiple axis plot of final segment attribute values over time;

b) connecting the plotted segment values creating a graph that resembles a cross-section of mountains and valleys;

c) tracing the plotted line;

d) resisting a maximum angle deviance of user input; and

e) effectively removing sharp peaks and crevices of said cross-sections, thereby minimizing content adjacency artifacts.

29. The method of claim 21, wherein the word significance value is determined by at least one word characteristic selected from the group consisting of whether the word is a noun or a pronoun, whether the word is a references to a character, a frequency of occurrence of the word in the audiovisual work, an identity of the character referenced, and a match of the word to a predetermined database word.