System and method for information seeking in a multimedia collection
First Claim
1. A computer-implemented method for information seeking in a multimedia collection of objects comprising:
- through a graphical user interface, receiving at least one of an input text query and an input image query;
providing for a user to select a subset of objects from a multimedia collection, at least some of the objects in the collection comprising first and second modalities, wherein the first and second modalities comprise a text modality and an image modality;
displaying representations of objects in the subset on a local map;
at each of a plurality of iterations;
providing for the user to annotate the text and image modalities of an object represented in the local map with a relevance label, wherein an object'"'"'s text modality and image modality are labeled independently;
automatically assigning a default forgetting factor for at least one of the first and second modalities, or providing for the user to select the forgetting factor for the at least one of the first and second modalities;
automatically assigning a default locality factor for at least one annotated object, or providing for the user to select a locality factor for the at least one annotated object;
with a computer processor, computing relevance scores for unlabeled objects in the collection, each relevance score taking into account labels applied to other objects, computed similarity measures between one of the unlabeled objects and the other objects in the collection, the locality factor selected by the user or the default locality factor, and the forgetting factor selected by the user or the default forgetting factor, the forgetting factor placing a greater weight on objects labeled in a more recent iteration;
modifying the local map based on the computed relevance scores; and
displaying the modified local map on a visual display.
6 Assignments
0 Petitions
Accused Products
Abstract
An apparatus and method facilitate combined query based searching with serendipitous browsing in a multimedia collection. A user selects objects to label from a local map, which may include representations of objects retrieved from the collection as being responsive to a text or image base query. The text and image portions of the object can be independently labeled. Unlabeled objects are scored and ranked based on the applied labels of labeled objects, which may take into account cross-media pseudo-relevance and user selectable (or default) parameters, such as a forgetting factor, which tends to place greater weight on more recently labeled objects, and a modality parameter, which laces greater weight on the modality (text, image, or hybrid) currently selected by the user. The local map is modified, based on the ranking, optionally after reranking of objects to improve the diversity of the displayed objects.
28 Citations
25 Claims
-
1. A computer-implemented method for information seeking in a multimedia collection of objects comprising:
-
through a graphical user interface, receiving at least one of an input text query and an input image query; providing for a user to select a subset of objects from a multimedia collection, at least some of the objects in the collection comprising first and second modalities, wherein the first and second modalities comprise a text modality and an image modality; displaying representations of objects in the subset on a local map; at each of a plurality of iterations; providing for the user to annotate the text and image modalities of an object represented in the local map with a relevance label, wherein an object'"'"'s text modality and image modality are labeled independently; automatically assigning a default forgetting factor for at least one of the first and second modalities, or providing for the user to select the forgetting factor for the at least one of the first and second modalities; automatically assigning a default locality factor for at least one annotated object, or providing for the user to select a locality factor for the at least one annotated object; with a computer processor, computing relevance scores for unlabeled objects in the collection, each relevance score taking into account labels applied to other objects, computed similarity measures between one of the unlabeled objects and the other objects in the collection, the locality factor selected by the user or the default locality factor, and the forgetting factor selected by the user or the default forgetting factor, the forgetting factor placing a greater weight on objects labeled in a more recent iteration; modifying the local map based on the computed relevance scores; and displaying the modified local map on a visual display. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 25)
-
-
19. An apparatus comprising:
-
a graphical user interface; memory which stores instructions for; receiving at least one of an input text query and an input image query through the graphical user interface; providing for a user to select a subset of objects from a multimedia collection, at least some of the objects in the collection comprising first and second modalities, wherein the first and second modalities comprise a text modality and an image modality; displaying representations of objects in the subset on a local map; at each of a plurality of iterations; providing for the user to annotate the text and image modalities of an object represented in the local map with a relevance label, wherein an object'"'"'s text modality and image modality are labeled independently; automatically assigning a default forgetting factor for at least one of the first and second modalities, or providing for the user to select the forgetting factor for the at least one of the first and second modalities; automatically assigning a default locality factor for at least one annotated object, or providing for the user to select a locality factor for the at least one annotated object; computing relevance scores for unlabeled objects in the collection, each relevance score taking into account labels applied to other objects, computed similarity measures between one of the unlabeled objects and the other objects in the collection, the locality factor selected by the user or the default locality factor, and the forgetting factor selected by the user or the default forgetting factor, the forgetting factor placing a greater weight on objects labeled in a more recent iteration; modifying the local map based on the computed relevance scores; and displaying the modified local map on a visual display; and a processor in communication with the memory for executing the instructions. - View Dependent Claims (20, 21, 22, 23, 24)
-
Specification