Object-based image search system and method

• US 8,055,103 B2
• Filed: 06/07/2007
• Issued: 11/08/2011
• Est. Priority Date: 06/08/2006
• Status: Active Grant

• Current Assignee: National Chiao Tung University (Government of The Republic of China)
• Original Assignee: National Chiao Tung University (Government of The Republic of China)
• Inventors: Xu, Yeong Yuh, Fu, Hsin Chia
• Primary Examiner(s): PERUNGAVOOR, SATHYANARAYA V
• Application Number:
  US11/759,911
• Publication Number:
  US 20070286531A1
• Time in Patent Office: 1,615 Days
• Field of Search: 382/305
• US Class Current: 382/305
• CPC Class Codes: G06F 16/532 Query formulation, e.g. graphical querying
• CPC Class Codes: G06V 10/40 Extraction of image or video features
• CPC Class Codes: G06V 10/757 Matching configurations of points or features
• CPC Class Codes: G06V 10/759 Region-based matching

Abstract

The present invention discloses an object-based image search system and method, whereby a user may visually specify query objects for retrieving relevant images without using any image-segmenting software tool. The method of the present invention comprises: specifying target feature points from sample images which are displayed in an image query interface and determining logic relationships of the target feature points; utilizing a feature-point checking program to correlate the target feature points with target objects; utilizing a feature-similarity calculating program to find out the images relevant to the target objects via the image feature parameters stored in a feature database; arranging and presenting the relevant images in sequence.

Claims

1. An image search method, which retrieves at least one relevant image from a plurality of candidate images via computerized calculations, wherein each said candidate image has at least one candidate object, and each said candidate object has a plurality of candidate-object feature parameters, comprising the following steps:

a. providing an image query interface, wherein said image query interface presents p sample images, each of said p sample images having a plurality of target features, and a user selects q target feature points defined by a subset of said target features within each of said p sample images, where p≧

1, and q≧

1;

b. locating q corresponding target objects from said p sample images according to said q target feature points, wherein each of said q target objects has a plurality of target-object feature parameters, which includes at least one target-object position parameter, at least one target-object color parameter, at least one target-object texture parameter and at least one target-object shape parameter;

c. performing similarity calculations between said q target objects and said plurality of said candidate images to retrieve images relevant to said q target objects, wherein each said candidate image has at least one candidate feature point defined by a plurality of candidate features within said candidate image, and one corresponding candidate object is located from said candidate image according to said candidate feature point, wherein said candidate object has a plurality of candidate-object feature parameters, which includes at least one candidate-object position parameter, at least one candidate-object color parameter, at least one candidate-object texture parameter and at least one candidate-object shape parameter, that cooperate with said target-object feature parameters of said q target objects to implement said similarity calculations, wherein said performing similarity calculations further comprises performing a weighted summation of the distance between said target-object feature parameters and said candidate-object feature parameters;

d. arranging said candidate images in sequence according to values obtained in said similarity calculations and outputting said candidate images having been arranged in sequence; and

wherein the relevant image from a plurality of candidate images is retrieved responsive to a user inputted text keyword or uploaded picture.

2. An image search method according to claim 1, wherein p≧

2.

3. An image search method according to claim 1, wherein said target objects of said sample images are combined by logic operations to meet search requirements.

4. An image search method according to claim 3 comprising similarity calculations between each said candidate object of said plurality of said candidate images and a plurality of said target objects combined by a logic operation.

5. An image search system, which retrieves at least one relevant image from a plurality of candidate images via computerized calculations and comprises a processor, a memory and a storage device, wherein said memory has an image search engine, and said storage device has an image database, and said image database contains a plurality of candidate images, and said processor operates said image search engine to realize the following mechanisms:

Mechanism A;

providing an image query interface, wherein said image query interface presents p sample images, each of said p sample images having a plurality of target features, and a user selects q target feature points defined by a subset of said target features within each of said p sample images, where p≧

1, and q≧

1;

Mechanism B;

locating q corresponding target objects from said p sample images according to said q target feature points, wherein each of said q target objects has a plurality of target-object feature parameters, which includes at least one target-object position parameter, at least one target-object color parameter, at least one target-object texture parameter and at least one target-object shape parameter;

Mechanism C;

performing similarity calculations between said q target objects and said plurality of said candidate images to retrieve images relevant to said q target objects, wherein each said candidate image has at least one candidate feature point defined by a plurality of candidate features within said candidate image, and one corresponding candidate object is located from said candidate image according to said candidate feature point, wherein said candidate object has a plurality of candidate-object feature parameters, which includes at least one candidate-object position parameter, at least one candidate-object color parameter, at least one candidate-object texture and at least one candidate-object shape parameter, that cooperate with said target-object feature parameters of said q target objects to implement said similarity calculations, wherein said performing similarity calculations further comprises performing a weighted summation of a distance between said target-object feature parameters and said candidate-object feature parameters;

Mechanism D;

arranging said candidate images in sequence according to values obtained in said similarity calculations and outputting said candidate images having been arranged in sequence; and

wherein the relevant image from a plurality of candidate images is retrieved responsive to a user inputted text keyword or uploaded picture.

6. An image search system according to claim 5, wherein p≧

2.

7. An image search system according to claim 5, wherein said target objects of said sample images are combined by logic operations to meet search requirements.

8. An image search system according to claim 7 comprising similarity calculations between each said candidate object of said plurality of said candidate images and a plurality of said target objects combined by a logic operation.