Computerized machine learning of interesting video sections
First Claim
1. A method comprising:
- receiving, by one or more computing devices, video data;
extracting, by at least one of the one or more computing devices, a plurality of features from the video data;
determining, by at least one of the one or more computing devices, a first set of feature values associated with the plurality of features, the first set of feature values for training a classifier and a scoring model;
determining, by at least one of the one or more computing devices, a second set of feature values based on applying the classifier to the video data;
training, by at least one of the one or more computing devices, the scoring model based on the first set of feature values and the second set of feature values;
using the scoring model to determine a plurality of desirability scores associated with the video data, wherein an individual desirability score indicative of video quality is associated with an individual video frame in the video data;
identifying video frames in the video data that have a desirability score above a predetermined threshold desirability score;
analyzing the video data to determine, in association with the video frames, changes in camera motion and changes in object motion; and
locating, based at least in part on the changes in camera motion and the changes in object motion, boundaries in the video data to produce one or more video segments, wherein;
an individual video segment includes at least one video frame that has the desirability score above the predetermined threshold desirability score, andthe locating the boundaries in the video data comprises determining that object motion intensity of a first video frame and object motion intensity of a second video frame differ by a predetermined threshold.
2 Assignments
0 Petitions
Accused Products
Abstract
This disclosure describes techniques for training models from video data and applying the learned models to identify desirable video data. Video data may be labeled to indicate a semantic category and/or a score indicative of desirability. The video data may be processed to extract low and high level features. A classifier and a scoring model may be trained based on the extracted features. The classifier may estimate a probability that the video data belongs to at least one of the categories in a set of semantic categories. The scoring model may determine a desirability score for the video data. New video data may be processed to extract low and high level features, and feature values may be determined based on the extracted features. The learned classifier and scoring model may be applied to the feature values to determine a desirability score associated with the new video data.
-
Citations
17 Claims
-
1. A method comprising:
-
receiving, by one or more computing devices, video data; extracting, by at least one of the one or more computing devices, a plurality of features from the video data; determining, by at least one of the one or more computing devices, a first set of feature values associated with the plurality of features, the first set of feature values for training a classifier and a scoring model; determining, by at least one of the one or more computing devices, a second set of feature values based on applying the classifier to the video data; training, by at least one of the one or more computing devices, the scoring model based on the first set of feature values and the second set of feature values; using the scoring model to determine a plurality of desirability scores associated with the video data, wherein an individual desirability score indicative of video quality is associated with an individual video frame in the video data; identifying video frames in the video data that have a desirability score above a predetermined threshold desirability score; analyzing the video data to determine, in association with the video frames, changes in camera motion and changes in object motion; and locating, based at least in part on the changes in camera motion and the changes in object motion, boundaries in the video data to produce one or more video segments, wherein; an individual video segment includes at least one video frame that has the desirability score above the predetermined threshold desirability score, and the locating the boundaries in the video data comprises determining that object motion intensity of a first video frame and object motion intensity of a second video frame differ by a predetermined threshold. - View Dependent Claims (2, 3, 4, 5, 6)
-
-
7. A system comprising:
-
memory; one or more processors; and one or more modules stored in the memory and executable by the one or more processors, the one or more modules including; an extracting module configured to extract features from video data and determine a first set of feature values based on the extracted features; a classifying module configured to apply a classifier to the first set of feature values to determine a second set of feature values and to use the second set of feature values to determine a probability that the video data belongs to at least one semantic category of a predefined set of semantic categories; a scoring module configured to apply a scoring model, based at least in part on the at least one semantic category, to the first set of feature values and the second set of feature values to determine a plurality of desirability scores for the video data, wherein an individual desirability score indicative of video quality is associated with an individual video frame in the video data; and a segmenting module configured to; identify video frames in the video data that have a desirability score above a predetermined threshold desirability score; analyze the video data to determine, in association with the video frames, changes in camera motion and changes in object motion; and locate, based at least in part on the changes in camera motion and the changes in object motion, boundaries in the video data to produce one or more video segments, wherein; an individual video segment includes at least one video frame that has the desirability score above the predetermined threshold desirability score, and the locating the boundaries in the video data comprises determining that object motion intensity of a first video frame and object motion intensity of a second video frame differ by a predetermined threshold. - View Dependent Claims (8, 9, 16)
-
-
10. One or more computer-readable storage media encoded with instructions that, when executed by a processor, perform acts comprising:
-
receiving video data including a plurality of video frames; extracting a plurality of features from individual video frames of the plurality of video frames to determine a first set of feature values associated with the individual video frames; applying a classifier to the first set of feature values to determine a second set of feature values associated with the individual video frames; using the second set of feature values to determine individual probabilities that the individual video frames belong to at least one semantic category of a predefined set of semantic categories; applying a scoring model to the first set of feature values and the second set of feature values to determine desirability scores associated with the individual video frames; identifying a subset of the plurality of video frames in the video data that have a desirability score above a predetermined threshold desirability score; analyzing the video data to determine, in association with the subset of video frames, changes in camera motion and changes in object motion; and locating, based at least in part on the changes in camera motion and the changes in object motion, boundaries in the video data to produce one or more video segments, wherein; an individual video segment includes at least one video frame that has the desirability score above the predetermined threshold desirability score, and the locating the boundaries in the video data comprises determining that object motion intensity of a first video frame and object motion intensity of a second video frame differ by a predetermined threshold. - View Dependent Claims (11, 12, 13, 14, 15, 17)
-
Specification