Identifying presentation styles of educational videos

• US 9,652,675 B2
• Filed: 07/23/2014
• Issued: 05/16/2017
• Est. Priority Date: 07/23/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Kannan, Anitha, Baker, Simon John
• Primary Examiner(s): RIES, LAURIE ANNE
• Application Number:
  US14/339,242
• Publication Number:
  US 20160026872A1
• Time in Patent Office: 1,028 Days
• CPC Class Codes: G06F 16/35 Clustering; Classification
• CPC Class Codes: G06F 16/735 Filtering based on additional data, e.g. user or group profiles
• CPC Class Codes: G06F 16/738 Presentation of query results
• CPC Class Codes: G06F 16/7837 using objects detected or recognised in the video content
• CPC Class Codes: G06F 16/784 the detected or recognised objects being people
• CPC Class Codes: G06F 16/7847 using low-level visual features of the video content
• CPC Class Codes: G06F 16/7867 using information manually generated, e.g. tags, keywords, comments, title and artist information, manually generated time, location and usage information, user ratings
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 20/41 Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items segmenting video sequences G06V20/49
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G09B 5/02 with visual presentation of the material to be studied, e.g. using film strip

Abstract

The presentation style of a video is identified. A set of features that represents the video is computed. A pre-learned video presentation style classifier is then used to weight each of the features in the set of features and determine a presentation style that is predominately employed in the video based on the weighting of the features.

Claims

1. A computer-implemented process for identifying the presentation style of a video, comprising the actions of:

using one or more computing devices that are in communication with each other via a computer network to perform the following process actions;

receiving the video;

computing a set of image features that are computed for each frame of the video independently, the image features comprising one or more of,

a low-contrast feature measuring pixel intensity statistics for the video according to a prescribed low-contrast threshold, or

a high-contrast feature measuring pixel intensity statistics for the video according to a prescribed high-contrast threshold, or

a zero-gradients feature measuring the amount of zero gradients that exist in the video, or

a low-gradients feature measuring the amount of weak but non-zero gradients that exist in the video, or

a high-gradients feature measuring the amount of strong gradients that exist in the video, or

a noise feature measuring the amount of pixel intensity noise that exists in the video; and

using a pre-learned video presentation style classifier to weight each of the image features in said set and determine a presentation style that is predominately employed in the video, said presentation style determination being based on the weighting of said features.

2. The process of claim 1, wherein the determined presentation style comprises a one of the presentation styles in a set of possible presentation styles comprising:

a rendered video class of presentation styles; and

a real-world video class of presentation styles.

3. The process of claim 2, wherein the video comprises an educational video, and the rendered video class of presentation styles comprises one or more of:

a rendered slide show presentation style;

or

a rendered slide show comprising a video of a presenter presentation style;

or

a rendered animation presentation style;

or

a rendered photographs presentation style;

or

a rendered hand-drawn slides presentation style.

4. The process of claim 2, wherein the video comprises an educational video, and the real-world video class of presentation styles comprises one or more of:

a natural video presentation style;

or

a video of an interview presentation style;

or

a video of handwriting on paper presentation style;

or

a video of projected slides presentation style;

or

a video of a whiteboard presentation style;

or

a video of a blackboard presentation style.

5. A computer-implemented process for identifying the presentation style of a video, comprising the actions of:

using one or more computing devices that are in communication with each other via a computer network to perform the following process actions;

receiving the video;

computing a set of face features that represent the video, wherein the face features are based on the detection of one or more faces in the video, the face features comprising one or more of,

a face detection feature measuring the percentage of frames in the video where just one face is detected, or

one or more moving face features each measuring, for each frame of the video where just one face is detected, whether or not said detected face is moving, or

a face not present feature measuring the length of the longest sequence of frames in the video where no face is detected, or

a face present feature measuring the length of the longest sequence of frames in the video where just one face is detected, or

a face size feature measuring, across the frames in the video where just one face is detected, the average size of said detected face; and

using a pre-learned video presentation style classifier to weight each of the face features in said set and determine a presentation style that is predominately employed in the video, said presentation style determination being based on the weighting of said features.

6. The process of claim 5, wherein the determined presentation style comprises a one of the presentation styles in a set of possible presentation styles comprising:

a rendered video class of presentation styles; and

a real-world video class of presentation styles.

7. The process of claim 6, wherein the video comprises an educational video, and the rendered video class of presentation styles comprises one or more of:

a rendered slide show presentation style;

or

a rendered slide show comprising a video of a presenter presentation style;

or

a rendered animation presentation style;

or

a rendered photographs presentation style;

or

a rendered hand-drawn slides presentation style.

8. The process of claim 6, wherein the video comprises an educational video, and the real-world video class of presentation styles comprises one or more of:

a natural video presentation style;

or

a video of an interview presentation style;

or

a video of handwriting on paper presentation style;

or

a video of projected slides presentation style;

or

a video of a whiteboard presentation style;

or

a video of a blackboard presentation style.

9. A computer-implemented process for identifying the presentation style of a video, comprising the actions of:

using one or more computing devices that are in communication with each other via a computer network to perform the following process actions;

receiving the video;

computing a set of motion features that represent the video, wherein the motion features are based on how the video changes from frame to frame, the motion features comprising one or more of,

frequency of motion features measuring how often motion occurs in the video, or

amount of motion features measuring how much motion takes place in the video, or

type of motion features specifying the type of motion that takes place in the video; and

using a pre-learned video presentation style classifier to weight each of the motion features in said set and determine a presentation style that is predominately employed in the video, said presentation style determination being based on the weighting of said features.

10. The process of claim 9, wherein the frequency of motion features comprise one or more of:

a motion frequency feature measuring the percentage of frames in the video where the magnitude of motion is greater than or equal to a prescribed motion frequency threshold;

or

another motion frequency feature measuring the percentage of frames in the video where the magnitude of motion is greater than or equal to another prescribed motion frequency threshold;

or

a motion present feature measuring the length of the longest sequence of frames in the video where there is motion;

or

a motion not present feature measuring the length of the longest sequence of frames in the video where there is no motion.

11. The process of claim 9, wherein the amount of motion features comprise one or more of:

a motion amount feature measuring the number of pixels in the video whose intensity changes between successive frames of the video according to a prescribed motion pixel threshold;

or

another motion amount feature measuring the number of pixels in the video whose intensity changes between successive frames of the video according to another prescribed motion pixel threshold.

12. The process of claim 9, wherein the type of motion features comprise one or more of:

a motion type feature (feat_mott1) given by the equation feat_mott1=Percentile_f (NRFlow(f), T_mott1), NRFlow(f) denoting the magnitude of non-rigid motion between a given frame (f) of the video and the immediately succeeding frame (f+1) of the video, T_mott1 denoting a prescribed motion type threshold, and Percentile_f sorting the values of NRFlow(f) across all of the frames of the video and then selecting the value of NRFlow(f) at the T_mott1 percentile;

or

another motion type feature (feat_mott2) given by the equation feat_mott2=Percentile_f (NRFlow(f)/OFlow(f), T_mott2), OFlow(f) denoting the magnitude of optical flow across frame f, T_mott2 denoting another prescribed motion type threshold, NRFlow(f)/OFlow(f) denoting the fraction of OFlow(f) that is non-rigid, and Percentile_f sorting the values of NRFlow(f)/OFlow(f) across all of the frames of the video and then selecting the value of NRFlow(f)/OFlow(f) at the T_mott2 percentile;

or

yet another motion type feature (feat_mott3) given by the equation feat_mott3=Percentile_f (OFRes(f),T_mott3), OFRes(f) denoting an optical flow residual indicating the degree to which changes between frame f and frame f+1 are due to the motion of scene elements in the video, T_mott3 denoting yet another prescribed motion type threshold, and Percentile_f sorting the values of OFRes(f) across all of the frames of the video and then selecting the value of OFRes(f) at the T_mott3 percentile.

13. The process of claim 9, wherein the determined presentation style comprises a one of the presentation styles in a set of possible presentation styles comprising:

a rendered video class of presentation styles; and

a real-world video class of presentation styles.

14. The process of claim 13, wherein the video comprises an educational video, and the rendered video class of presentation styles comprises one or more of:

a rendered slide show presentation style;

or

a rendered slide show comprising a video of a presenter presentation style;

or

a rendered animation presentation style;

or

a rendered photographs presentation style;

or

a rendered hand-drawn slides presentation style.

15. The process of claim 13, wherein the video comprises an educational video, and the real-world video class of presentation styles comprises one or more of:

a natural video presentation style;

or

a video of an interview presentation style;

or

a video of handwriting on paper presentation style;

or

a video of projected slides presentation style;

or

a video of a whiteboard presentation style;

or

a video of a blackboard presentation style.