Relevance maximizing, iteration minimizing, relevance-feedback, content-based image retrieval (CBIR)

• US 7,546,293 B2
• Filed: 07/17/2006
• Issued: 06/09/2009
• Est. Priority Date: 03/30/2001
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Zhu, Xingquan, Su, Zhong, Zhang, Hong-Jiang
• Primary Examiner(s): AL HASHEMI, SANA A
• Application Number:
  US11/458,057
• Publication Number:
  US 20060248044A1
• Time in Patent Office: 1,058 Days
• Field of Search: 707/2, 707/3, 707/5, 707/6, 707/10, 707/104.1
• US Class Current: 1/1
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 18/2178 based on feedback of a supervisor
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 18/24155 Bayesian classification
• CPC Class Codes: G06F 18/40 Software arrangements specially adapted for pattern recognition, e.g. user interfaces or toolboxes therefor
• CPC Class Codes: G06V 10/761 Proximity, similarity or dissimilarity measures
• CPC Class Codes: G06V 10/7784 based on feedback from supervisors
• CPC Class Codes: G06V 10/945 User interactive design; Environments; Toolboxes
• CPC Class Codes: Y10S 707/99931 Database or file accessing
• CPC Class Codes: Y10S 707/99932 Access augmentation or optimizing
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99934 Query formulation, input preparation, or translation
• CPC Class Codes: Y10S 707/99935 Query augmenting and refining, e.g. inexact access
• CPC Class Codes: Y10S 707/99936 Pattern matching access
• CPC Class Codes: Y10S 707/99943 Generating database or data structure, e.g. via user interface
• CPC Class Codes: Y10S 707/99945 Object-oriented database structure processing
• CPC Class Codes: Y10S 707/99948 Application of database or data structure, e.g. distributed, multimedia, or image

Abstract

An implementation of a technology, described herein, for relevance-feedback, content-based image retrieval minimizes the number of iterations for user feedback regarding the semantic relevance of exemplary images while maximizing the resulting relevance of each iteration. One technique for accomplishing this is to use a Bayesian classifier to treat positive and negative feedback examples with different strategies. In addition, query refinement techniques are applied to pinpoint the users'"'"' intended queries with respect to their feedbacks. These techniques further enhance the accuracy and usability of relevance feedback. This abstract itself is not intended to limit the scope of this patent. The scope of the present invention is pointed out in the appending claims.

Claims

1. A method for improving iterative results of Content-Based Image Retrieval (CBIR) using relevance feedback, the method comprising:

obtaining by a CBIR system configured to facilitate relevance maximizing, iteration minimizing, relevance-feedback CBIR a set of positive feedback images and separately a set of negative feedback images which are stored in a computer-readable storage medium and identified by a user, wherein the set of positive feedback images and the set of negative feedback images are disjoint, and the set of positive feedback images are those images deemed semantically relevant and the set of negative feedback images are those deemed semantically less relevant;

within a feature space, moving, by the CBIR system, a positive candidate image towards the set of positive feedback images by adjusting distance metrics of the positive candidate image, the positive candidate image having similar low-level features as those of the set of positive feedback images,

within a feature space, distancing, by the CBIR system, a negative candidate image from the set of positive feedback images by adjusting distance metrics of the negative candidate image, the negative candidate image having similar low-level features as those of the set of negative feedback images;

constructing, by the CBIR system, a Bayesian classifier of a positive feedback image by positive candidate images.

2. One or more computer-readable storage media having computer-executable instructions that, when executed by a computer, performs a method for improving iterative results of Content-Based Image Retrieval (CBIR) using relevance feedback, the method comprising:

obtaining a set of positive feedback images and separately a set of negative feedback images which are stored in a computer-readable storage medium and identified by a user, wherein the set of positive feedback images and the set of negative feedback images are disjoint, and the set of positive feedback images are those images deemed semantically relevant and the set of negative feedback images are those deemed semantically less relevant;

within a feature space, moving a positive candidate image towards the set of positive feedback images by adjusting distance metrics of the positive candidate image, the positive candidate image having similar low-level features as those of the set of positive feedback images,

within a feature space, distancing a negative candidate image from the set of positive feedback images by adjusting distance metrics of the negative candidate image, the negative candidate image having similar low-level features as those of the set of negative feedback images;

employing a Bayesian decision boundary function to determine the probability of an image being a positive candidate image.

3. A method for improving iterative results of Content-Based Image Retrieval (CBIR) using relevance feedback, the method comprising:

obtaining by a CBIR system configured to facilitate relevance maximizing, iteration minimizing, relevance-feedback CBIR a set of positive feedback images and separately a set of negative feedback images which are stored in a computer-readable storage medium and identified by a user, wherein the set of positive feedback images and the set of negative feedback images are disjoint, and the set of positive feedback images are those images deemed semantically relevant and the set of negative feedback images are those deemed semantically less relevant;

within a feature space, moving, by the CBIR system, a positive candidate image towards the set of positive feedback images by adjusting distance metrics of the positive candidate image, the positive candidate image having similar low-level features as those of the set of positive feedback images, wherein the moving further comprises;

normalizing, by the CBIR system, features of an image within a feature space;

initializing, by the CBIR system, parameters of the classifier;

updating, by the CBIR system, the parameters using the features of the new positive feedback images;

calculating, by the CBIR system, distances based upon a Bayesian decision boundary function;

sorting, by the CBIR system, images based upon calculated distances, wherein the sort is performed as if no negative feedback images exist;

within a feature space, distancing, by the CBIR system, a negative candidate image from the set of positive feedback images by adjusting distance metrics of the negative candidate image, the negative candidate image having similar low-level features as those of the set of negative feedback images;

constructing, by the CBIR system, a Bayesian classifier of a positive feedback image by positive candidate images.

4. One or more computer-readable storage media having computer-executable instructions that, when executed by a computer, performs a method for improving iterative results of Content-Based Image Retrieval (CBIR) using relevance feedback, the method comprising:

receiving a sample image input;

presenting retrieval results based on distances to the sample image input;

obtaining positive feedback images and separately negative feedback images which are stored in a computer-readable storage medium and identified by a user, wherein the positive feedback images and the negative feedback images are disjoint;

with the positive feedback images, updating parameters of a Bayesian classifier;

according to the negative feedback images, applying a dibbling process where distances from images in the retrieval results to the negative feedback images are incremented;

presenting new retrieval results based on the Bayesian classifier and the incremented distances to the negative feedback images.

5. One or more computer-readable storage media as recited in claim 4, wherein the dibbling process punishes those images near negative examples in features space without affecting others in the feature space.

6. One or more computer-readable storage media as recited in claim 4 further comprising:

retrieving images from an image database, wherein such retrieving is based upon the results of the updating and the applying;

presenting updated retrieval results of the retrieving;

awaiting additional feedback based upon the updated retrieval results.