CHARACTER-BASED AUTOMATED SHOT SUMMARIZATION

• US 20130138435A1
• Filed: 01/26/2013
• Published: 05/30/2013
• Est. Priority Date: 10/27/2008
• Status: Active Grant

• Current Assignee: Frank Elmo Weber
• Original Assignee: Frank Elmo Weber
• Inventors: Weber, Frank Elmo
• Granted Patent: US 8,812,311 B2
• Time in Patent Office: Days
• US Class Current: 704/231
• CPC Class Codes: G06F 16/345 Summarisation for human users
• CPC Class Codes: G06F 16/739 in form of a video summary, e.g. the video summary being a video sequence, a composite still image or having synthesized frames
• CPC Class Codes: G06F 16/7844 using original textual content or text extracted from visual content or transcript of audio data
• CPC Class Codes: G06F 40/10 Text processing natural language analysis G06F40/20; semantic analysis G06F40/30; processing or translation of natural language G06F40/40
• CPC Class Codes: G06V 20/47 Detecting features for summarising video content
• CPC Class Codes: G10H 1/46 Volume control
• CPC Class Codes: G10H 2210/076 for extraction of timing, tempo; Beat detection
• CPC Class Codes: G10L 15/00 Speech recognition G10L17/00 takes precedence
• CPC Class Codes: G10L 25/57 for processing of video signals
• CPC Class Codes: G10L 25/63 for estimating an emotional state
• CPC Class Codes: G10L 25/87 Detection of discrete points within a voice signal

Abstract

Methods, devices, systems and tools are presented that allow the summarization of text, audio, and audiovisual presentations, such as movies, into less lengthy forms. High-content media files are shortened in a manner that preserves important details, by splitting the files into segments, rating the segments, and reassembling preferred segments into a final abridged piece. Summarization of media can be customized by user selection of criteria, and opens new possibilities for delivering entertainment, news, and information in the form of dense, information-rich content that can be viewed by means of broadcast or cable distribution, “on-demand” distribution, internet and cell phone digital video streaming, or can be downloaded onto an iPod™ and other portable video playback devices.

Claims

1. A method for summarizing an audiovisual presentation, comprising:

a) providing an audiovisual presentation having a first temporal length,

i) a plurality of shot segments, each shot segment comprising a segment component;

ii) a plurality of secondary segments selected from the group comprising a dialogue segment, a non-dialogue segment, a non-dialogue segment with music segment, and an audience reaction segment;

b) rating each shot segment by automatically determining a shot segment significance value for each shot segment, the method comprising;

determining at least one segment component significance value for at least one segment component characteristic including determining whether the segment component is a noun, pronoun, reference to a story character, a frequency of occurrence of the segment component, a match of the segment component to a predetermined database of segment components; and

combining the at least one segment component significance value to determine the shot segment significance value;

c) optionally determining the secondary significance value of one or more of the secondary segments wherein the secondary significance value weights the significance value of the shot segment;

d) receiving a user generated time significance value threshold;

e) using the significance value threshold to select the shot segment if the shot segment significance value exceeds the significance value threshold, and delete the shot segment if the shot segment significance value is below the significance value threshold;

f) combining each shot segment with the shot segment significance value exceeding the significance value threshold into a summarized version of the audiovisual presentation; and

g) providing the summarized version of the audiovisual presentation, the audiovisual presentation having a second temporal length.

2. The method of claim 1, wherein the segment component significance values are determined by the identity of a story character referenced.

3. The method of claim 2, wherein the segment component is the reference to a character and the segment component significance value is determined by a frequency of the character'"'"'s appearance in the audiovisual presentation and a frequency of use of a pronoun referring to the character in the audiovisual presentation.

4. The method of claim 1, wherein the shot significance value is weighted by a perceived presence of stress and the perceived presence of stress is determined by assessing a character communication parameter selected from the group consisting of a volume of speech, a tone of speech, a rate of speech, and a facial expression.

5. The method of claim 1 further comprising:

a) receiving a playback time request within which to play the summarized audiovisual presentation; and

b) automatically calculating the shot significance value threshold so that the summarized audiovisual presentation fits within the playback time request.

6. The method of claim 1, wherein the audiovisual presentation further comprises an original shot segment, wherein a predetermined first portion of the original shot segment and a predetermined last portion of the original shot segment is selected to include in the summarized version of the audiovisual presentation and a predetermined middle portion of the original shot segment is excluded from the summarized version of the audiovisual presentation, wherein the ratio of the combined length of the included portions to the length of the original shot segment is equal to the ratio of the second temporal length to the first temporal length.

7. The method of claim 1, wherein the shot segment significance threshold is determined automatically when a user determines a percentage of truncation of the audiovisual presentation.

8. The method of claim 1, wherein the audiovisual presentation further comprises an original shot segment, wherein a portion of the original shot segment is selected to include in the summarized version of the audiovisual presentation, wherein the ratio of the length of the included portion to the length of the original shot segment is equal to the ratio of the second temporal length to the first temporal length.

9. The method of claim 1, wherein the audiovisual presentation further comprises an original shot segment, wherein a portion of the original shot segment is selected to include in the summarized version of the audiovisual presentation, wherein the length of the included portion is determined by the shot significance value.

10. The method of claim 1, wherein a single cinematic shot is comprised of at least one shot segment.

11. The method of claim 1, wherein segment components of a shot segment are determined by data evidencing the entrance or exit of characters in an audiovisual presentation.

12. The method of claim 1, wherein segment components of a shot segment are determined by data evidencing the entrance or exit of objects in an audiovisual presentation.

13. The method of claim 1, wherein segment components of a shot segment are determined by data evidencing the abrupt changes in speed of characters in an audiovisual presentation.

14. The method of claim 1, wherein segment components of a shot segment are determined by data evidencing the abrupt flashes of light in an audiovisual presentation.

15. The method of claim 1, wherein segment components of a shot segment are determined by data evidencing the changes in scenery in an audiovisual presentation.

16. The method of claim 1, wherein segment components of a shot segment are determined by data evidencing the presence and volume of sound in an audiovisual presentation.