Semi-automatic annotation of multimedia objects
First Claim
1. A method comprising:
- identifying, in response to a search query, first multimedia objects having an associated keyword that matches a keyword in the search query, and identifying second multimedia objects that have content features similar to those of the first multimedia objects, wherein the content features do not match the keyword in the search query;
presenting the first and second multimedia objects to a user;
monitoring feedback from the user as to which of the first and second multimedia objects are relevant to the search query;
maintaining associations between the keywords and the multimedia objects, the associations being weighted to indicate how relevant the keywords are to the multimedia objects;
adjusting the weights of the associations based on the user'"'"'s feedback, wherein the adjusting comprises increasing a weight of an association between the keyword and a particular multimedia object that is deemed relevant by the user or/and decreasing a weight of an association between the keyword and a particular multimedia object that is deemed irrelevant by the user; and
annotating one or more of the second multimedia objects, which are deemed relevant by the user, with the keyword.
2 Assignments
0 Petitions
Accused Products
Abstract
A multimedia object retrieval and annotation system integrates an annotation process with object retrieval and relevance feedback processes. The annotation process annotates multimedia objects, such as digital images, with semantically relevant keywords. The annotation process is performed in background, hidden from the user, as the user conducts normal searches. The annotation process is “semi-automatic” in that it utilizes both keyword-based information retrieval and content-based image retrieval techniques to automatically search for multimedia objects, and then encourages users to provide feedback on the retrieved objects. The user identifies objects as either relevant or irrelevant to the query keywords and based on this feedback, the system automatically annotates the objects with semantically relevant keywords and/or updates associations between the keywords and objects. As the retrieval-feedback-annotation cycle is repeated, the annotation coverage and accuracy of future searches continues to improve.
-
Citations
27 Claims
-
1. A method comprising:
-
identifying, in response to a search query, first multimedia objects having an associated keyword that matches a keyword in the search query, and identifying second multimedia objects that have content features similar to those of the first multimedia objects, wherein the content features do not match the keyword in the search query;
presenting the first and second multimedia objects to a user;
monitoring feedback from the user as to which of the first and second multimedia objects are relevant to the search query;
maintaining associations between the keywords and the multimedia objects, the associations being weighted to indicate how relevant the keywords are to the multimedia objects;
adjusting the weights of the associations based on the user'"'"'s feedback, wherein the adjusting comprises increasing a weight of an association between the keyword and a particular multimedia object that is deemed relevant by the user or/and decreasing a weight of an association between the keyword and a particular multimedia object that is deemed irrelevant by the user; and
annotating one or more of the second multimedia objects, which are deemed relevant by the user, with the keyword. - View Dependent Claims (2, 3, 4, 5, 6)
-
-
7. A method comprising:
-
identifying, in response to a search query, first multimedia objects having an associated keyword that matches a keyword in the search query, and identifying second multimedia objects that have content features similar to those of the first multimedia objects, wherein the content features do not match the keyword in the search query;
presenting the first and second multimedia objects to a user;
monitoring feedback from the user as to which of the first and second multimedia objects are relevant to the search query;
maintaining associations between the keywords and the multimedia objects, the associations being weighted to indicate how relevant the keywords are to the multimedia objects;
adjusting the weights of the associations based on the user'"'"'s feedback, wherein the adjusting comprises increasing a weight of an association between the keyword and a particular multimedia object that is deemed relevant by the user or/and decreasing a weight of an association between the keyword and a particular multimedia object that is deemed irrelevant by the user; and
annotating one or more of the second multimedia objects, which are deemed relevant by the user, with the keyword. - View Dependent Claims (8, 9, 10, 11, 12, 13)
-
-
14. A method comprising:
-
identifying, in response to a search query, first multimedia objects having an associated keyword that matches a keyword in the search query, and identifying second multimedia objects that have content features similar to those of the first multimedia objects, wherein the content features do not match the keyword in the search query;
presenting the first and second multimedia objects to a user;
monitoring feedback from the user as to which of the multimedia objects are relevant;
annotating one or more of the multimedia objects based on the user'"'"'s feedback, with a keyword;
in an event that a particular multimedia object is deemed relevant by the user and not yet annotated with the keyword, adding the keyword to the particular multimedia object; and
in an event that the particular multimedia object is deemed relevant by the user and is already annotated with the keyword, strengthening an association between the keyword and the particular multimedia object. - View Dependent Claims (15, 16, 17, 18, 19)
-
-
20. A system comprising:
-
an information retrieval unit to retrieve multimedia objects from a database based on a search query first multimedia objects having an associated keyword that matches a keyword in the search query, and identifying second multimedia objects that have content features similar to those of the first multimedia objects wherein the content features do not match the keyword in the search query;
a relevance feedback unit to capture a user'"'"'s feedback as to whether the multimedia objects are relevant to the search query; and
an annotation unit to annotate, with the keyword, the second multimedia objects based on the user'"'"'s feedback, wherein the search query comprises a keyword-based search query having at least one keyword and in an event that a particular multimedia object is deemed relevant by the user and is already annotated with the keyword, the annotation unit strengthens an association between the keyword and the particular multimedia object, and/or in an event that a particular multimedia object is deemed irrelevant by the user and is already annotated with the keyword, removing the keyword from the particular multimedia object, and/or in an event that a particular multimedia object is deemed irrelevant by the user and is already annotated with the keyword, weakening an association between the keyword and the particular multimedia object. - View Dependent Claims (21, 22, 23, 24, 25, 26, 27)
-
Specification