Dynamic feature weighting

• US 10,210,179 B2
• Filed: 11/18/2008
• Issued: 02/19/2019
• Est. Priority Date: 11/18/2008
• Status: Active Grant

• Current Assignee: R2 Solutions LLC (Acacia Research Corporation)
• Original Assignee: Excalibur IP, LLC (Acacia Research Corporation)
• Inventors: van Zwol, Roelof, van Leuken, Reinier H.
• Primary Examiner(s): Thomas, Ashish
• Assistant Examiner(s): Mahmood, Rezwanul
• Application Number:
  US12/273,197
• Publication Number:
  US 20100125568A1
• Time in Patent Office: 3,745 Days
• Field of Search: 707723, 707765, 707769, 707999001, 707999003
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06V 10/40 Extraction of image or video features

Abstract

This disclosure describes systems and methods for identifying and ranking relevant and diverse image search results in response to a query. An optimum set of features is identified for every query. The optimum set of features can be selected from a predefined set of features and can be selected based on a variance across features derived from an initial set of objects returned in response to the query. The optimum set of features can then be used to re-rank the initial set of objects or to search for a second set of objects and rank the second set of objects.

Claims

1. A method comprising the steps of:

receiving, by a processor, an initial set of images related to a query via a network, the initial set of images obtained from a first ranking that is based on a first determination of relevance to the query;

selecting, by the processor, a plurality of highest ranked images from the initial set of images to form a subset of selected images;

receiving, by the processor, a set of features from the images of the subset based on characteristics of the images of the subset, the characteristics comprising at least a variance for each feature of the set of features across the subset of selected images;

analyzing, by the processor, variance for each feature of the set of features across the subset of selected images;

identifying, by the processor, features of the plurality of highest ranked images in the image subset having greater values of variance relative to other features in the set of features so as to distinguish features from the feature set;

creating, by the processor, a second determination of relevance to the query providing more relevant and more diverse image search results than the first determination, the creating comprising including the identified features having greater values of variance into an optimum subset of features, the second determination of relevance for each image to the query being determined via weighting each feature of the optimum subset of features based on the variance for each feature and performing a summation of the weighted features;

performing, by the processor, a second ranking of the initial set of images from the first ranking by using the optimum subset of features according to the second determination of relevance to the query such that the images from the second ranking are more relevant to the query, and more diverse, than the subset of selected images from the first ranking; and

returning, by the processor, the images from the second ranking in response to the query.

2. The method of claim 1, wherein greater weight is assigned to features having greater variance.

3. The method of claim 1, wherein the images are ranked based on a first determination of diversity.

4. The method of claim 1, wherein the second relevance of each image to the query is determined via the following equation:

5. The method of claim 1, wherein the query is text.

6. The method of claim 1, wherein the query is an image.

7. The method of claim 1, wherein re-ranking the set of images includes performing a second query using the optimum subset of features.

8. The method of claim 1, wherein features include at least one or more of the following:

a color layout, a color histogram, scalable color, an edge histogram, an edge directivity descriptor, and tamura.

9. The method of claim 1, wherein features include non-visual features.

10. The method of claim 9, wherein non-visual features are extracted from text associated with an image, image metadata, image spatial information, query spatial information, image temporal information, query temporal information, social characteristics associated with the image or the user, or a user profile.

11. A non-transitory computer readable storage medium tangibly comprising computer readable instructions for:

receiving an initial set of images related to a query via a network, the initial set of images obtained from a first ranking that is based on a first determination of relevance to the query;

selecting a plurality of highest ranked images from the initial set of images to form a subset of selected images;

receiving a set of features from the images of the subset based on characteristics of the images of the subset, the characteristics comprising at least a variance for each feature of the set of features across the images of the subset;

analyzing variance for each feature of the set of features across the subset of selected images;

identifying features of the plurality of highest ranked images in the image subset having greater values of variance relative to other features in the set of features so as to distinguish features from the feature set;

creating a second determination of relevance to the query providing more relevant and more diverse image search results than the first determination, the creating comprising including the identified features having greater values of variance into an optimum subset of features, the second determination of relevance for each image to the query being determined via weighting each feature of the optimum subset of features based on the variance for each feature and performing a summation of the weighted features;

performing a second ranking of the initial set of images from the first ranking by using the optimum subset of features according to the second determination of relevance to the query such that the images from the second ranking are more relevant to the query, and more diverse, than the subset of selected images from the first ranking; and

returning the images from the second ranking in response to the query.

12. A system comprising:

a processor; and

a non-transitory computer readable storage medium storing thereon program logic for execution by the processor, the program logic comprising;

receiving logic executed by the processor for receiving an initial set of images selected for its relevance to a query input from a first ranking that is based on a first determination comprising a predefined set of image features;

selection logic executed by the processor for selecting a plurality of the highest ranked images from the initial set of images to form a subset of selected images;

identification logic executed by the processor for analyzing variance for each feature of the predefined set of image features across the subset of selected images for identifying those image features of the plurality of highest ranked images in the image subset having greater values of variance relative to other features in the set of features so as to distinguish features from the feature set;

creation logic executed by the processor for creating a second determination of relevance to the query providing more relevant and more diverse image search results than the first determination, the creating comprising including the identified features having greater values of variance into an optimum subset of features, the second determination of relevance for each image to the query being determined via weighting each feature of the optimum subset of features based on the variance for each feature and performing a summation of the weighted features;

determination logic executed by the processor for determining a second ranking of the initial set of images from the first ranking by using the optimum subset of features according to the second determination of relevance to the query such that the images from the second ranking are more relevant to a query input, and more diverse, than the subset of selected images from the first ranking; and

communication logic executed by the processor for returning the images from the second ranking.

13. The system of claim 12 further comprising:

query logic executed by the processor for performing a second query for image search results based on the optimum subset of features.