Determining importance of scenes based upon closed captioning data

• US 8,763,023 B1
• Filed: 03/08/2013
• Issued: 06/24/2014
• Est. Priority Date: 03/08/2013
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Goetz, Jeromey Russell
• Primary Examiner(s): Lonsberry, Hunter B
• Application Number:
  US13/789,718
• Time in Patent Office: 473 Days
• Field of Search: 725 19- 20, 725 44- 47
• US Class Current: 725/20
• CPC Class Codes: G06F 16/7844 using original textual content or text extracted from visual content or transcript of audio data
• CPC Class Codes: H04H 60/377 Scene
• CPC Class Codes: H04N 21/233 Processing of audio elementary streams monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/251 Learning process for intelligent management, e.g. learning user preferences for recommending movies details of learning user preferences for the retrieval of video data in a video database G06F16/739; computer systems using learning methods G06N3/08
• CPC Class Codes: H04N 21/4394 involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/4884 for displaying subtitles
• CPC Class Codes: H04N 21/8549 Creating video summaries, e.g. movie trailer retrieval in video databases by using presentations in form of a video summary G06F16/739
• CPC Class Codes: H04N 7/0882 for the transmission of character code signals, e.g. for teletext circuits for the digital non-picture data signal H04N7/035

Abstract

Disclosed are various embodiments for identifying importance of scenes or moments in video content relative to one another. Closed captioning data is extracted from a video content feature. A textual analysis of the closed captioning data is performed. The importance level of scenes can be ranked with respect to one another.

Claims

1. A non-transitory computer-readable medium embodying a program executable in at least one computing device, comprising:

code that obtains a video content feature, the video content feature associated with closed captioning data;

code that performs an analysis of spoken dialog in the closed captioning data in a plurality of scenes of the video content feature;

code that performs an analysis of non-dialog spoken content in the closed captioning data in the plurality of scenes of the video content feature;

code that performs an analysis of text in the closed captioning data that is associated with unspoken content in the video content feature; and

code that ranks a plurality of scenes of the video content feature according to an importance level, the importance level of a scene based at least in part upon the analyses of the spoken dialog, the non-dialog spoken content and the unspoken content.

2. The non-transitory computer-readable medium of claim 1, wherein the analysis of the spoken dialog further comprises:

code that determines whether a portion of the spoken dialog contains dialog identified by a user in a video information site as being associated with an important scene; and

code that modifies the importance level of a scene corresponding to the portion of the spoken dialog.

3. The non-transitory computer-readable medium of claim 1, wherein the analysis of the non-dialog spoken content further comprises:

code that determines whether the a portion of the non-dialog spoken content corresponds to narration; and

code that modifies the importance level of a scene corresponding to the portion of the non-dialog spoken content.

4. A system, comprising:

at least one computing device; and

a video content analysis application executable in the at least one computing device, the video content analysis application comprising;

logic that extracts closed captioning data from a video content feature;

logic that identifies a plurality of scenes in the video content feature;

logic that performs a textual analysis on the closed captioning data; and

logic that ranks an importance level of the plurality of scenes based at least in part upon the textual analysis of the closed captioning data.

5. The system of claim 4, wherein the logic that identifies a plurality of scenes in the video content feature further comprises logic that performs scene change detection within the video content feature.

6. The system of claim 4, wherein the ranks the importance level of the plurality of scenes based at least in part upon the textual analysis of the closed captioning data further comprises logic that ranks the plurality of scenes based at least in part upon a volume of dialog occurring within each of the plurality of scenes.

7. The system of claim 4, wherein the logic that performs the textual analysis on the closed captioning data further comprises:

logic that identifies text corresponding to a portion of dialog from the video content feature as being associated with an important scene from the video content feature;

logic that identifies the text corresponding to a portion of dialog from the video content feature in the closed captioning data; and

logic that modifies the importance level of a scene corresponding to the portion of dialog from the video content feature.

8. The system of claim 7, wherein the logic that identifies text corresponding to a portion of dialog from the video content feature as being associated with the important scene further comprises logic that determines whether the text corresponds to at least one of a plurality user identified quotations associated with the video content feature from a data store in communication with the at least one computing device.

9. The system of claim 7, wherein the logic that identifies text corresponding to a portion of dialog from the video content feature as being associated with an important scene further comprises logic that performs a sentiment analysis on the closed captioning data.

10. The system of claim 4, wherein the logic that performs the textual analysis on the closed captioning data further comprises:

logic that retrieves a synopsis associated with the video content feature from a data store in communication with the at least one computing device;

logic that identifies text from the synopsis in the closed captioning data; and

logic that modifies the importance level of a scene corresponding to the text from the synopsis in the closed captioning data.

11. The system of claim 7, wherein the logic that performs the textual analysis on the closed captioning data further comprises:

logic that ranks a plurality of performers appearing within the video content feature according to prominence;

logic that identifies at least one respective scene containing a highest ranked performer;

logic that modifies the importance level of the at least one respective scene including the highest ranked performer.

12. The system of claim 11, wherein the logic that identifies at least one respective scene containing a highest ranked performer further comprises logic that identifies a name of a character of the performer within the closed captioning data.

13. The system of claim 11, wherein the logic that ranks the plurality of performers appearing within the video content feature according to prominence further comprises logic that ranks the plurality of performers based at least in part upon an amount of dialog in the closed captioning data that is attributable to each of the respective performers.

14. The system of claim 11, wherein the logic that ranks the plurality of performers appearing within the video content feature according to prominence further comprises logic that queries an external network site containing information about the plurality of performers appearing within video content feature.

15. The system of claim 4, wherein the logic that ranks the importance level of the plurality of scenes based at least in part upon the textual analysis of the closed captioning data further comprises logic that tags the video content feature according to the ranking.

16. A method, comprising:

obtaining, by at least one computing device, a video content feature associated with closed captioning data;

extracting, by the at least one computing device, the closed captioning data from the video content feature; and

identifying, by the at least one computing device, an importance level of a plurality of moments from within the video content feature based at least in part upon an analysis of the closed captioning data.

17. The method of claim 16, wherein identifying the importance level of the plurality of moments from within the video content feature further comprises:

identifying, by the at least one computing device, non-dialog spoken content in the closed captioning data;

identifying, by the at least one computing device, a moment corresponding to the non-dialog spoken content; and

modifying, by the at least one computing device, the importance level of the moment corresponding to the non-dialog spoken content.

18. The method of claim 16, wherein identifying the importance level of the plurality of moments from within the video content feature further comprises:

identifying, by the at least one computing device, at least one character within the closed captioning data that corresponds to narration;

identifying, by the at least one computing device, a moment within the video content feature corresponding to the narration; and

modifying, by the at least one computing device, an importance level of the moment corresponding to the narration.

19. The method of claim 16, wherein identifying the importance level of the plurality of moments from within the video content feature further comprises:

identifying, by the at least one computing device, non-spoken content within the closed captioning data;

determining, by the at least one computing device, whether the non-spoken content corresponds to a keyword; and

modifying, by the at least one computing device, an importance level of the moment corresponding to the non-spoken content.

20. The method of claim 19, wherein identifying the non-spoken content within the closed captioning data further comprises:

identifying a leading character indicating a beginning of non-spoken content within the closed captioning data; and

identifying a termination character indicating an ending of non-spoken content within the closed captioning data.

21. The method of claim 19, wherein the keyword is designated as indicative of an important scene within the video content feature.

22. The method of claim 16, wherein identifying the importance level of the plurality of moments from within the video content feature further comprises:

identifying, by the at least one computing device, an indication of music within the closed captioning data; and

modifying, by the at least one computing device, the importance level of the moment based at least in part upon the indication of music within the closed captioning data.

23. The method of claim 22, further comprising:

identifying, by the at least one computing device, a song title within the closed captioning data corresponding to the music; and

modifying, by the at least one computing device, the importance level of the moment based at least in part upon the song title.

24. The method of claim 23, further comprising modifying, by the at least one computing device, the importance level of the moment based at least in part upon a popularity of a song corresponding to the song title.