×

System and method for image annotation and multi-modal image retrieval using probabilistic semantic models

  • US 7,814,040 B1
  • Filed: 01/24/2007
  • Issued: 10/12/2010
  • Est. Priority Date: 01/31/2006
  • Status: Active Grant
First Claim
Patent Images

1. A computer-implemented method of retrieving media, comprising:

  • (a) creating a probabilistic framework relating media types within a mixed media work to implicit concepts;

    (b) creating an index of a set of media based on implicit concepts within the probabilistic framework;

    (c) receiving a query expressed in the form of a media exemplar;

    (d) determining a set of concepts expressed in the media exemplar;

    (e) searching the index of the set of media for elements representing similar implicit concepts to those expressed in the media exemplar; and

    (f) outputting at least one representation or identifier of the elements representing similar implicit concepts to those expressed in the media exemplar;

    wherein said outputting comprises outputting a representation or identifiers of a plurality of elements, further comprising;

    ranking the plurality of elements based on at least a similarity of the respective element to implicit concepts expressed in the media exemplar;

    wherein said determining comprises probabilistically determining a set of semantic concepts inherent in the media exemplar, based on correlations of features in respective multimedia works having predetermined semantic concepts associated therewith;

    further comprising determining a concept vector for the media exemplar;

    wherein the probabilistic framework comprises a Bayesian model for associating words with an image having visual features, comprising a hidden concept layer which connects a visual feature layer and a word layer which is discovered by fitting a generative model to a training set comprising images having the visual features and annotation words, wherein the conditional probabilities of the visual features and the annotation words given a hidden concept class are determined based on an Expectation-Maximization (EM) based iterative learning procedure.

View all claims
  • 2 Assignments
Timeline View
Assignment View
    ×
    ×