×

Audio-based annotation of video

  • US 9,715,902 B2
  • Filed: 06/05/2014
  • Issued: 07/25/2017
  • Est. Priority Date: 06/06/2013
  • Status: Active Grant
First Claim
Patent Images

1. A computer-implemented method, comprising:

  • obtaining a plurality of user-provided content items that include respective audio information and video information associated with a type of annotation item;

    training a set of classifiers using the plurality of user-provided content items to determine a relationship between features of the plurality of user-provided content items and the type of annotation item, the set of classifiers configured to identify a customized list of a set of annotation items;

    receiving a content item that includes audio information and video information, the audio information being time synchronized with the video information;

    extracting the audio information from the content item;

    determining an acoustic pattern that characterizes a portion of the audio information by performing audio analysis on the audio information, wherein the acoustic pattern is associated with a temporal location in the audio information;

    determining a corresponding temporal location in the video information based at least in part on the temporal location of the acoustic pattern;

    analyzing a portion of the video information at the corresponding temporal location using the set of classifiers to determine a set of matching vectors, individual matching vectors of the set of matching vectors including weighted annotation items from the customized list of the set of annotation items;

    determining a weighted summation of the weighted annotation items of the set of matching vectors to generate a merged matching vector; and

    selecting an annotation item from the customized list of the set of annotation items from the merged matching vector associated with a highest weight, wherein the annotation item characterizes the video information at the corresponding temporal location.

View all claims
  • 4 Assignments
Timeline View
Assignment View
    ×
    ×