×

Video annotation method by integrating visual features and frequent patterns

  • US 20080059872A1
  • Filed: 03/05/2007
  • Published: 03/06/2008
  • Est. Priority Date: 09/05/2006
  • Status: Active Grant
First Claim
Patent Images

1. A video annotation method by integrating visual features and frequent patterns, comprising:

  • providing a plurality of fundamental words;

    providing an annotated video clip, wherein said annotated video clip is composed of a plurality of first shots, and each of said first shots is composed of a plurality of first frames, and each of said first shots is corresponding to at least one first annotation word of said fundamental words;

    performing a data preprocessing step, said data preprocessing step comprising;

    selecting a plurality of first critical frames respectively with respect to said first shots from said first frames of each of said first shots;

    dividing each of said first critical frames into a plurality of first image blocks;

    respectively extracting low-level features of said first image blocks of each of said first sots, thereby obtaining a plurality of first block feature vectors of each of said first critical frames;

    respectively extracting low-level features of each of said first critical frames, thereby obtaining a plurality of first feature vectors of said first shots;

    performing a grouping step for dividing said first feature vectors into a plurality of shot groups, wherein said shot groups have a plurality of identification codes respectively;

    corresponding said first feature vectors to said identification codes respectively; and

    combining said identification codes of said shot groups as at least one first scene;

    building a statistical model by using said first block feature vectors and said at least one first annotation word with respect to each of said first shots in accordance with a Gaussian Mixtures Model and conditional probabilities, wherein said statistical model has a statistical probability list used for indicating the respective appearing probabilities of said fundamental words corresponding to said first block feature vectors of each of said first shots;

    building a sequential model, comprising;

    finding frequent patterns of said shot groups in said first scene in accordance with a continuous relevance algorithm, thereby obtaining a plurality of first sequential rules, wherein said first sequential rules are the sequential transaction combinations of any two identification codes arbitrarily selected in each of said at least one first scene; and

    building said sequential model in accordance with each of said first sequential rules and said at least one first annotation word corresponding thereto, wherein said sequential model has a sequential probability list used for indicating the respective appearing probabilities of said fundamental words corresponding to each of said first sequential rules;

    performing a predicting stage for inputting a second shot desired to be annotated into said statistical model and said sequential model, thereby obtaining a keyword statistical probability list and a keyword sequential probability list, wherein said keyword statistical probability list is used for indicating the respective appearing probabilities of said fundamental words corresponding to a plurality of second block feature vectors of said second shot, and said keyword sequential probability list is used for indicating the respective appearing probabilities of said fundamental words corresponding to a plurality of second sequential rules of said second shot, and said second shot belongs to a second scene and is composed of a plurality of second frames.

View all claims
  • 1 Assignment
Timeline View
Assignment View
    ×
    ×