Representation and retrieval of images using context vectors derived from image information elements

• US 7,072,872 B2
• Filed: 06/14/2004
• Issued: 07/04/2006
• Est. Priority Date: 09/20/1993
• Status: Expired due to Fees

• Current Assignee: Fair Isaac Corporation
• Original Assignee: Fair Isaac Corporation
• Inventors: Hecht-Neilsen, Robert, Caid, William R.
• Primary Examiner(s): Knight, Anthony
• Assistant Examiner(s): Holmes, Michael B.
• Application Number:
  US10/868,538
• Publication Number:
  US 20040249774A1
• Time in Patent Office: 750 Days
• Field of Search: 706/14
• US Class Current: 706/14
• CPC Class Codes: G06F 16/30 of unstructured textual data document management systems G06F16/93
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06V 10/451 with interaction between the filter responses, e.g. cortical complex cells

Abstract

Image features are generated by performing wavelet transformations at sample points on images stored in electronic form. Multiple wavelet transformations at a point are combined to form an image feature vector. A prototypical set of feature vectors, or atoms, is derived from the set of feature vectors to form an “atomic vocabulary.” The prototypical feature vectors are derived using a vector quantization method, e.g., using neural network self-organization techniques, in which a vector quantization network is also generated. The atomic vocabulary is used to define new images. Meaning is established between atoms in the atomic vocabulary. High-dimensional context vectors are assigned to each atom. The context vectors are then trained as a function of the proximity and co-occurrence of each atom to other atoms in the image. After training, the context vectors associated with the atoms that comprise an image are combined to form a summary vector for the image. Images are retrieved using a number of query methods, e.g., images, image portions, vocabulary atoms, index terms. The user'"'"'s query is converted into a query context vector. A dot product is calculated between the query vector and the summary vectors to locate images having the closest meaning. The invention is also applicable to video or temporally related images, and can also be used in conjunction with other context vector data domains such as text or audio, thereby linking images to such data domains.

Claims

1. A computer-implemented method for training context vectors for objects within documents, comprising the steps of:

for each of a plurality of objects, generating a plurality of feature vectors from object data of the object;

for each object, associating each of the object'"'"'s feature vectors with a context vector;

for each object, aligning each of the context vectors of the object using a context vector of at least one word included in a document containing the object; and

aligning each of the context vectors of the object by adjusting the object context vector to be more similar to the summary vector of the at least one word included in the document containing the image.

2. The method of claim 1, wherein the object is any of:

text, image, image data, and one or more image features.

3. The method of claim 1, wherein the step of associating each of the object'"'"'s feature vectors with a context vector further comprises the step of:

initializing the context vectors to be substantially orthogonal to each other.

4. The method of claim 1, wherein the context vectors are oriented in a vector space, and wherein axes of the vector space not associated with selected objects or object features.

5. A computer-implemented method for training context vectors for objects within documents, comprising the steps of:

providing a plurality of word context vectors, each context vector having an orientation in a vector space, wherein words having similar meaning have context vectors with similar orientations in the vector space;

providing a plurality of object context vectors, each object context vector associated with a feature vector, each feature vector derived from object data of at least one object, each object context vector having an orientation in the vector space;

for each document containing an object, aligning the object context vectors associated with the feature vectors derived from the object, with a summary vector derived from context vectors of selected words contained in the document; and

wherein aligning the object context vectors with a summary vector derived from context vectors of selected words contained in the document by adjusting the object context vector to be more similar to the summary vector of the selected words included in the document.

6. The method of claim 5, wherein the object is any of:

text, image, image data, and one or more image features.

7. The method of claim 5, wherein the context vectors are oriented in the vector space, and wherein axes of the vector space not associated with selected terms or image features.

8. A computer-implemented method for retrieving records having different object types, the method comprising:

providing a plurality of first records, each first record having a first object type;

for each of the first record having the first object type, deriving from elements of the first record a context vector, the context vector having an orientation in a vector space;

providing a plurality of second records, each second record having a second object type;

for each of the second records having the second object type, deriving from elements of the second record a context vector, the context vector having an orientation in the vector space;

receiving a query, comprising at least one element of the first object type;

deriving a query context vector from the query; and

retrieving at least one second record having a context vector similar to the query context vector.

9. The computer-implemented method of claim 8, wherein the object is any of a first media type and a second media type.