System for creative image navigation and exploration

• US 8,775,424 B2
• Filed: 01/26/2010
• Issued: 07/08/2014
• Est. Priority Date: 01/26/2010
• Status: Expired due to Fees

• Current Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Skaff, Sandra, Marchesotti, Luca, Colombino, Tommaso, Fucs, Ana, Csurka, Gabriela, Bressan, Marco, Wazaefi, Yanal
• Primary Examiner(s): Yen, Syling
• Application Number:
  US12/693,795
• Publication Number:
  US 20110184950A1
• Time in Patent Office: 1,624 Days
• Field of Search: 707/737, 707/769, 707999003-999006, 715/810, 382/305
• US Class Current: 707/737
• CPC Class Codes: G06F 16/54 Browsing; Visualisation therefor
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 20/70 Labelling scene content, e.g. deriving syntactic or semantic representations

Abstract

A system and method for assisting a user in navigation of an image dataset are disclosed. The method includes receiving a user'"'"'s text query, retrieving images responsive to the query from an image dataset, providing for receiving the user'"'"'s selection of a first feature selected from a set of available first features via a graphical user interface, providing for receiving the user'"'"'s selection of a second feature selected from a set of available second features different from the first features via the graphical user interface, and displaying at least some of the retrieved images on the graphical user interface. The displayed images are arranged, e.g., grouped, according to levels and/or combinations of levels of the user-selected first and second features.

Claims

1. A method for assisting a user in navigation of an image dataset comprising:

training a separate classifier for each of a plurality of emotional features, each emotional feature representing a pair of opposing emotions, using labels of labeled images and descriptors extracted from the labeled images;

receiving a user'"'"'s text query;

retrieving images responsive to the text query from a dataset of images, the images in the dataset being associated with emotional features output by the trained classifiers;

providing for receiving the user'"'"'s selection of a first feature selected from a set of available features, the set of features including the emotional features and aesthetic features;

providing for receiving the user'"'"'s selection of a second feature selected from the set of available features; and

displaying at least some of the retrieved images on the graphical user interface contemporaneously in an arrangement, whereby the arrangement of images reflects different levels of the first feature where only a first feature is selected and reflects different combinations of levels of the user-selected first and second features where first and second features are selected, the emotional features each including at least three levels,

wherein when first and second features are selected, the displaying includes displaying images in a two-dimensional array on the graphical user interface, wherein the first feature increases in level in a first dimension of the two dimensional array and the second feature increases in level in a second dimension of the two dimensional array, whereby positions of images in the array reflect respective combinations of feature levels.

2. The method of claim 1, wherein the displaying includes displaying at least some of the retrieved images on the graphical user interface contemporaneously in groups, whereby each displayed group of images corresponds to a different combination of levels of user-selected first and second features.

3. The method of claim 1, further comprising displaying, on the graphical user interface, a first selector for selection of the first feature from a first set of features and a second selector for selecting the second feature from a second set of features.

4. The method of claim 3, wherein the first features are aesthetic features and the second features are emotional features.

5. The method of claim 3, further comprising providing for a user to select a third feature from a third set of features, whereby the displayed images are grouped according to the selected at least two of the first, second, and third features.

6. The method of claim 2, wherein the displayed groups of images are ordered according to values of the selected features.

7. The method of claim 2, wherein each group of images in the display differs in a combination of quantized values of the first and second features from each of the other groups.

8. The method of claim 1, wherein for each feature in the set of features, a plurality of quantization levels corresponding to different values of the feature is provided and each of the images in the dataset of images is associated with a respective one of the quantization levels for each of the features.

9. The method of claim 8, wherein for each feature in the set of features, there are at least three quantization levels.

10. The method of claim 1, wherein the providing for receiving the user'"'"'s selections of the first and second features includes providing for receiving the user'"'"'s selections via the graphical user interface.

11. The method of claim 1, further comprising retrieving at least one of nearest neighbors and near duplicate images for a user-selected image or user-selected group of images and displaying the at least one of nearest neighbors and near duplicate images on the graphical user interface.

12. The method of claim 1, wherein the images comprise photographs.

13. The method of claim 1, further comprising automatically classifying at least a portion of the images in the dataset according to content, and wherein the retrieving of images responsive to a text query is based on the content classifications of the images.

14. The method of claim 1, further comprising displaying a document with a placeholder for accepting an image contemporaneously with a user-selected group of the images, whereby a user can select an image from the displayed group to be inserted in the placeholder.

15. The method of claim 1, wherein the displayed images are thumbnail images derived from corresponding images in the stored dataset.

16. The method of claim 1, wherein at least one of the first and second features is a color feature and the other of the first and second features is an emotion feature.

17. A computer program product comprising a non-transitory medium encoding instructions, which when executed by a computer, perform a method for assisting a user in navigation of an image dataset, the method comprising:

training a separate classifier for each of a plurality of emotional features, each emotional feature representing a pair of opposing emotions, using labels of labeled images and descriptors extracted from the labeled images;

receiving a user'"'"'s text query;

retrieving images responsive to the text query from a dataset of images, the images in the dataset being associated with emotional features output by the trained classifiers;

providing for receiving the user'"'"'s selection of a first feature selected from a set of available features, the set of features including the emotional features and aesthetic features;

providing for receiving the user'"'"'s selection of a second feature selected from the set of available features; and

displaying at least some of the retrieved images on the graphical user interface contemporaneously in an arrangement, whereby the arrangement of images reflects different levels of the first feature where only a first feature is selected and reflects different combinations of levels of the user-selected first and second features where first and second features are selected, the emotional features each including at least three levels,

wherein when first and second features are selected, the displaying includes displaying images in a two-dimensional array on the graphical user interface, wherein the first feature increases in level in a first dimension of the two dimensional array and the second feature increases in level in a second dimension of the two dimensional array, whereby positions of images in the array reflect respective combinations of feature levels.

18. A navigation system for assisting a user in navigation of an image dataset comprising:

memory which stores instructions for;

training a separate classifier for each of a plurality of emotional features, each emotional feature representing a pair of opposing emotions, using labels of labeled images and descriptors extracted from the labeled images,

receiving a user'"'"'s text query,

retrieving images responsive to the text query from a dataset of images, the images in the dataset being associated with emotional features output by the trained classifiers,

providing for receiving the user'"'"'s selection of a first feature selected from a set of available features, the set of features including the emotional features and aesthetic features,

providing for receiving the user'"'"'s selection of a second feature selected from the set of available features, and

displaying at least some of the retrieved images on the graphical user interface contemporaneously in an arrangement, whereby the arrangement of images reflects different levels of the first feature where only a first feature is selected and reflects different combinations of levels of the user-selected first and second features where first and second features are selected, the emotional features each including at least three levels,

wherein when first and second features are selected, the displaying includes displaying images in a two-dimensional array on the graphical user interface, wherein the first feature increases in level in a first dimension of the two dimensional array and the second feature increases in level in a second dimension of the two dimensional array, whereby positions of images in the array reflect respective combinations of feature levels; and

a processor in communication with the memory for executing the instructions.