LEARNING HIGHLIGHTS USING EVENT DETECTION

• US 20170323178A1
• Filed: 07/21/2017
• Published: 11/09/2017
• Est. Priority Date: 12/08/2010
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Kwatra, Vivek, Gargi, Ullas, Sargin, Mehmet Emre, Tang, Henry Hao
• Granted Patent: US 10,867,212 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 2218/12 Classification; Matching
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 20/10 using kernel methods, e.g. support vector machines [SVM]
• CPC Class Codes: G06V 10/40 Extraction of image or video features
• CPC Class Codes: G06V 10/507 Summing image-intensity values; Histogram projection analysis
• CPC Class Codes: G06V 20/42 of sport video content
• CPC Class Codes: G06V 20/44 Event detection

Abstract

A highlight learning technique is provided to detect and identify highlights in sports videos. A set of event models are calculated from low-level frame information of the sports videos to identify recurring events within the videos. The event models are used to characterize videos by detecting events within the videos and using the detected events to generate an event vector. The event vector is used to train a classifier to identify the videos as highlight or non-highlight.

Claims

1. A computer-implemented method for identifying sports highlights within videos, the method comprising:

receiving, using a hardware processor, at least a portion of a video content item of a sporting event;

extracting, using the hardware processor, from the portion of the video content item, a plurality of features associated with one or more frames of the portion of the video content item;

determining, using the hardware processor, whether a feature of the plurality of features matches a previously identified event of a plurality of previously identified events, wherein each of the plurality of previously identified events is an identifiable pattern of features within video content items;

in response to determining that the feature matches the previously identified event, determining, using the hardware processor, whether the portion of the video content item contains a sports highlight; and

in response to determining that the portion of the video content item contains the sports highlight, designating, using the hardware processor, the portion of the video content item as containing the sports highlight.

2. The method of claim 1, further comprising:

identifying a searchable keyword for the portion of the video content item in response to determining that the portion of the video content item contains the sports highlight; and

updating a tag associated with the video content item with the searchable keyword.

3. The method of claim 1, further comprising:

identifying a searchable keyword for the portion of the video content item in response to determining that the portion of the video content item contains the sports highlight; and

placing the portion of the video content item in a category for sports highlights based on the searchable keyword.

4. The method of claim 1, further comprising classifying the portion of the video content item using a plurality of highlight classifiers to generate a classification result, wherein the portion of the video content item is determined to contain the sports highlight based on the classification result.

5. The method of claim 1, further comprising generating an event vector for each event detected in the portion of the video content item.

6. The method of claim 1, further comprising associating statistics recorded in-time with the portion of the video content item.

7. The method of claim 1, wherein the plurality of features extracted from the portion of the video content item include visual low-level frame-based features.

8. A computer-implemented system for identifying sports highlights within videos, the system comprising:

a memory; and

a hardware processor that, when executing computer executable instructions stored in the memory, is configured to;

receive at least a portion of a video content item of a sporting event;

extract, from the portion of the video content item, a plurality of features associated with one or more frames of the portion of the video content item,

determine whether a feature of the plurality of features matches a previously identified event of a plurality of previously identified events, wherein each of the plurality of previously identified events is an identifiable pattern of features within video content items;

in response to determining that the feature matches the previously identified event, determine whether the portion of the video content item contains a sports highlight; and

in response to determining that the portion of the video content item contains the sports highlight, designate the portion of the video content item as containing the sports highlight.

9. The system of claim 8, wherein the hardware processor is further configured to:

identify a searchable keyword for the portion of the video content item in response to determining that the portion of the video content item contains the sports highlight; and

update a tag associated with the video content item with the searchable keyword.

10. The system of claim 8, wherein the hardware processor is further configured to:

identify a searchable keyword for the portion of the video content item in response to determining that the portion of the video content item contains the sports highlight; and

place the portion of the video content item in a category for sports highlights based on the searchable keyword.

11. The system of claim 8, wherein the hardware processor is further configured to classify the portion of the video content item using a plurality of highlight classifiers to generate a classification result, wherein the portion of the video content item is determined to contain the sports highlight based on the classification result.

12. The system of claim 8, wherein the hardware processor is further configured to generate an event vector for each event detected in the portion of the video content item.

13. The system of claim 8, wherein the hardware processor is further configured to associate statistics recorded in-time with the portion of the video content item.

14. The system of claim 8, wherein the plurality of features extracted from the portion of the video content item include visual low-level frame-based features.

15. A non-transitory computer-readable medium containing computer executable instructions that, when executed by a processor, cause the processor to perform a method for identifying sports highlights within videos, the method comprising:

receiving at least a portion of a video content item of a sporting event;

extracting, from the portion of the video content item, a plurality of features associated with one or more frames of the portion of the video content item;

determining whether a feature of the plurality of features matches a previously identified event of a plurality of previously identified events, wherein each of the plurality of previously identified events is an identifiable pattern of features within video content items;

in response to determining that the feature matches the previously identified event, determining whether the portion of the video content item contains a sports highlight; and

in response to determining that the portion of the video content item contains the sports highlight, designating the portion of the video content item as containing the sports highlight.

16. The non-transitory computer-readable medium of claim 15, wherein the method further comprises:

identifying a searchable keyword for the portion of the video content item in response to determining that the portion of the video content item contains the sports highlight; and

updating a tag associated with the video content item with the searchable keyword.

17. The non-transitory computer-readable medium of claim 15, wherein the method further comprises:

identifying a searchable keyword for the portion of the video content item in response to determining that the portion of the video content item contains the sports highlight; and

placing the portion of the video content item in a category for sports highlights based on the searchable keyword.

18. The non-transitory computer-readable medium of claim 15, wherein the method further comprises classifying the portion of the video content item using a plurality of highlight classifiers to generate a classification result, wherein the portion of the video content item is determined to contain the sports highlight based on the classification result.

19. The non-transitory computer-readable medium of claim 15, wherein the method further comprises generating an event vector for each event detected in the portion of the video content item.

20. The non-transitory computer-readable medium of claim 15, wherein the method further comprises associating statistics recorded in-time with the portion of the video content item.

21. The non-transitory computer-readable medium of claim 15, wherein the plurality of features extracted from the portion of the video content item include visual low-level frame-based features.