Derivative media content systems and methods

• US 10,499,121 B2
• Filed: 01/09/2018
• Issued: 12/03/2019
• Est. Priority Date: 01/09/2018
• Status: Active Grant

• Current Assignee: NBC Universal Media LLC (Comcast Corporation)
• Original Assignee: NBC Universal Media LLC (Comcast Corporation)
• Inventors: Beckett, William, Gupta, Gaurav, Markova, Milena
• Primary Examiner(s): Le, Rong
• Application Number:
  US15/866,082
• Publication Number:
  US 20190215578A1
• Time in Patent Office: 693 Days
• CPC Class Codes: G06F 16/685 using automatically derived transcript of audio data, e.g. lyrics speech recognition G10L15/00
• CPC Class Codes: G06F 16/739 in form of a video summary, e.g. the video summary being a video sequence, a composite still image or having synthesized frames
• CPC Class Codes: G06F 16/783 using metadata automatically derived from the content
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: H04N 21/23109 by placing content in organized collections, e.g. EPG data repository details of retrieval of video data and associated meta data in video databases G06F16/739
• CPC Class Codes: H04N 21/233 Processing of audio elementary streams monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/4314 for fitting data in a restricted space on the screen, e.g. EPG data in a rectangular grid
• CPC Class Codes: H04N 21/8126 involving additional data, e.g. news, sports, stocks, weather forecasts
• CPC Class Codes: H04N 21/8133 specifically related to the content, e.g. biography of the actors in a movie, detailed information about an article seen in a video program
• CPC Class Codes: H04N 21/8405 represented by keywords
• CPC Class Codes: H04N 21/845 Structuring of content, e.g. decomposing content into time segments
• CPC Class Codes: H04N 21/8456 by decomposing the content in the time domain, e.g. in time segments
• CPC Class Codes: H04N 21/8547 involving timestamps for synchronizing content
• CPC Class Codes: H04N 21/8549 Creating video summaries, e.g. movie trailer retrieval in video databases by using presentations in form of a video summary G06F16/739

Abstract

A system for generating video clips includes a video processing system configured to receive a video file from a video capture system, and to create video clips based on significant moments identified in the video file. The video processing system includes a file storage and database system configured to store the video file, and storing an ontology and keywords associated with themes that are pertinent to the overall theme of the video file. The video processing system also includes a natural language processing (NLP) moments module configured to identify themes contained in the video file using the ontology and the keywords, and to identify time codes associated with the identified themes. The video processing system also includes a video clip generator configured to generate video clips based on the identified time codes.

Claims

1. A method for generating video clips, comprising:

obtaining a set of timecoded words from speech of a video;

generating word groupings from the set of timecoded words using natural language processing (NLP), the generating comprising associating a theme with each word grouping based on a semantics model;

filtering the word groupings based on statistical relevance;

identifying expressions having the filtered word groupings;

grouping the expressions according to the theme of the word grouping associated with each expression;

generating new time codes for clip generation by identifying correlations between the grouped expressions and the video, comprising;

determining a start timecode of the new time codes by locating a punctuation mark occurring immediately prior to a particular grouped expression including a particular word grouping, and using the location of the punctuation mark as the start timecode for the theme associated with the particular word grouping; and

generating a video clip using the new time codes.

2. The method of claim 1, wherein generating the word groupings from the set of timecoded words comprises generating the word groupings from different sections of the set of timecoded words, the different sections corresponding to time-contiguous portions of the video.

3. The method of claim 1, wherein generating the word groupings comprises:

identifying keywords from a keyword database within the set of timecoded words; and

associating themes with groups of words encompassing each identified keyword using the semantics model to generate the word groupings.

4. The method of claim 1, wherein filtering the word groupings based on their statistical relevance comprises determining a frequency and a duration of the word groupings, and determining whether the frequency and the duration of the word groupings are within a frequency range and a duration range.

5. The method of claim 1, wherein generating the new time codes for clip generation comprises determining valid timecode boundaries for the video clip.

6. The method of claim 1, wherein obtaining the set of timecoded words from the speech of the video comprises deriving the set of timecoded words from the speech in timecoded caption metadata of the video.

7. The method of claim 1, further comprising processing audio associated with the video to convert speech in the video to text using a domain specific language model to produce the set of timecoded words corresponding to the video.

8. A method for generating video clips, comprising:

obtaining a set of timecoded words from speech of a video;

identifying seeding keywords from a keyword database within the set of timecoded words;

generating word groupings by identifying additional keywords from words surrounding the seeding keywords, wherein identifying the additional keywords comprises;

using natural language processing (NLP) to identify themes associated with the words surrounding the seeding keywords, and

determining if the words surrounding the seeding keywords have a common theme with the seeding keywords;

determining a statistical relevance for each word grouping to identify statistically relevant word groupings;

determining start timecodes and end timecodes for themes corresponding to the locations of the statistically relevant word groupings in the set of timecoded words, wherein determining the start timecodes comprises locating a respective punctuation mark occurring immediately prior to each of the statistically relevant word groupings and using the locations of each respective punctuation mark as the start timecodes for the themes; and

generating a video clip based on the determined start timecodes and end timecodes for the themes.

9. The method of claim 8, wherein determining the statistical relevance of the word groupings of interest comprises determining a frequency and a duration of the word groupings, and determining whether the frequency and duration of the word groupings are within a frequency range and a duration range.

10. The method of claim 8, wherein determining the statistical relevance of the word groupings of interest comprises determining a frequency and a duration of a theme of the word groupings, and determining whether the frequency and the duration of the theme are within a frequency range and a duration range.

11. The method of claim 8, wherein generating word groupings further comprises grouping words with the seeding keywords and the additional keywords in response to determining that the words have a common theme with the seeding keywords and the additional keywords.

12. The method of claim 11, wherein the words are not grouped with the seeding keywords if another theme is identified in a time between the words and the seeding keywords according to the timeframe of the video.

13. The method of claim 8, wherein the words surrounding the seeding keywords surround the seeding keywords in time according to the timeframe of the video.

14. A method for generating video clips, comprising:

obtaining a set of timecoded words from speech of a video;

identifying seeding keywords from a keyword database within the set of timecoded words;

generating word groupings by identifying additional keywords from words surrounding the seeding keywords, wherein identifying the additional keywords comprises;

using natural language processing (NLP) to identify themes associated with the words surrounding the seeding keywords,

determining if the words surrounding the seeding keywords have a common theme with the seeding keywords, and

grouping words with the seeding keywords and the additional keywords in response to determining that the words have a common theme with the seeding keywords and the additional keywords, wherein the words are not grouped with the seeding keywords if another theme is identified in a time between the words and the seeding keywords according to the timeframe of the video;

determining a statistical relevance for each word grouping to identify statistically relevant word groupings;

determining start timecodes and end timecodes for themes corresponding to the locations of the statistically relevant word groupings in the set of timecoded words; and

generating a video clip based on the determined start timecodes and end timecodes for the themes.

15. The method of claim 14, further comprising, for each statistically relevant word grouping, identifying a beginning of an expression containing the respective word grouping, locating a first time code associated with the beginning of the expression, and using the located first time code as a starting point for the video clip.

16. The method of claim 15, wherein identifying the beginning of the expression further comprises locating a punctuation mark occurring immediately prior to the statistically relevant word grouping, and using the punctuation mark as the location of the first timecode associated with the beginning of the expression.

17. The method of claim 14, wherein determining the statistical relevance of the word groupings of interest comprises determining a frequency and a duration of the word groupings, and determining whether the frequency and duration of the word groupings are within a frequency range and a duration range.

18. The method of claim 14, wherein determining the statistical relevance of the word groupings of interest comprises determining a frequency and a duration of a theme of the word groupings, and determining whether the frequency and the duration of the theme are within a frequency range and a duration range.

19. The method of claim 14, wherein obtaining the set of timecoded words from the speech of the video comprises deriving the set of timecoded words from the speech in timecoded caption metadata of the video.

20. The method of claim 14, wherein obtaining the set of timecoded words comprises deriving the set of timecoded words from audio associated with the video using a domain specific language model.