Perceptual similarity image retrieval
First Claim
1. A method of indexing images according to visual characteristics of the images, the method comprising the steps of:
- receiving an array of pixel representations that represent an image;
processing the array of pixel representations to determine a plurality of spots associated with the image, each spot comprising a connected region of the image having selected visual characteristics that vary spatially within a selected tolerance;
generating a plurality of spot descriptors, each spot descriptor indicating the visual characteristics of a respective spot; and
using the plurality of spot descriptors as an index entry for the image.
7 Assignments
0 Petitions
Accused Products
Abstract
A system and method indexes an image database by partitioning an image thereof into a plurality of cells, combining the cells into intervals and then spots according to perceptual criteria, and generating a set of spot descriptors that characterize the perceptual features of the spots, such as their shape, color and relative position within the image. The shape preferably is a derivative of the coefficients of a Discrete Fourier Transform (DFT) of the perimeter trace of the spot. The set of spot descriptors forms as an index entry for the spot. This process repeated for the various images of the database. To search the index, a key comprising a set of spot descriptors for a query image is generated and compared according to a perceptual similarity metric to the entries of the index. The metric determines the perceptual similarity that the features of the query image match those of the indexed image. The search results are presented as a scored list of the indexed images. A wide variety of image types can be indexed and searched, including: bi-tonal, gray-scale, color, “real scene” originated, and artificially generated images. Continuous-tone “real scene” images such as digitized still pictures and video frames are of primary interest. There are stand alone and networked embodiments. A hybrid embodiment generates keys locally and performs image and index storage and perceptual comparison on a network or web server.
145 Citations
24 Claims
-
1. A method of indexing images according to visual characteristics of the images, the method comprising the steps of:
-
receiving an array of pixel representations that represent an image; processing the array of pixel representations to determine a plurality of spots associated with the image, each spot comprising a connected region of the image having selected visual characteristics that vary spatially within a selected tolerance; generating a plurality of spot descriptors, each spot descriptor indicating the visual characteristics of a respective spot; and using the plurality of spot descriptors as an index entry for the image. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10)
-
-
11. A method of generating a plurality of spots associated with an image, a spot comprising a connected region of the image and having substantially similar visual characteristics, the method comprising:
- the steps of
receiving an array of pixel representations; partitioning the array of pixel representations to generate an array of cells, the array have first and second axial directions, each cell comprising a like-sized connected region of the image; determining a plurality of visual characteristics of each cell; joining cells connected along the first axial direction to form a plurality of disjoint intervals, each interval having visual characteristics that vary by at most a first tolerance; and joining intervals connected along the second axial direction to form at least one spot, each spot having visual characteristics that vary by at most a second tolerance. - View Dependent Claims (12, 13, 14, 15, 16, 17, 18, 19)
- the steps of
-
20. A method of generating a description of the shape of a region in a digital image, the method comprising the steps of:
-
generating an approximation of the shape of the region, said approximation comprising a plurality of connected like-sized cells that intersect the region; generating a sequence of coordinates along a boundary curve of the approximation; transforming the sequence of coordinates to a spectral domain; and processing a plurality of lower frequency spectral coefficients of the transformed sequence to generate the description of the region. - View Dependent Claims (21)
-
-
22. A computer readable medium carrying instructions executable by a computer to perform a method of indexing images according to visual characteristics of the images, the method comprising the steps of:
-
receiving an array of pixel representations that represent an image; processing the array of pixel representations to determine a plurality of spots associated with the image, each spot comprising a connected region of the image having selected visual characteristics that vary spatially within a selected tolerance; generating a plurality of spot descriptors, each spot descriptor indicating the visual characteristics of a respective spot; and using the plurality of spot descriptors as an index entry for the image.
-
-
23. A computer readable medium carrying instructions executable by a computer to perform a method of generating a plurality of spots associated with an image, each spot comprising a connected region of the image having substantially uniform visual characteristics, the method comprising the steps of:
-
receiving an array of pixel representations; partitioning the array of pixel representations to generate an array of cells, the array having first and second axial directions, each cell comprising a like-sized connected region of the image; determining a plurality of visual characteristics of each cell; joining cells connected along the first axial direction to form a plurality of disjoint intervals, each interval having visual characteristics that vary spatially by at most a first tolerance; and joining intervals connected along the second axial direction to form at least one spot, each spot having visual characteristics that vary spatially by at most a second tolerance.
-
-
24. A computer readable medium carrying instructions executable by a computer to perform a method of generating a description of a shape of a region in a digital image, the method comprising the steps of:
-
generating an approximation of the shape of the region, said approximation comprising a plurality of connected like-sized cells that intersect the region; generating a sequence of coordinates along a boundary curve of the approximation; transforming the sequence of coordinates to a spectral domain; and processing a plurality of lower frequency spectral coefficients of the transformed sequence to generate the description of the region.
-
Specification