System and method for measuring image similarity based on semantic meaning

• US 20060143176A1
• Filed: 02/21/2006
• Published: 06/29/2006
• Est. Priority Date: 04/15/2002
• Status: Active Grant

• Current Assignee: Wrp IP Management, LLC
• Original Assignee: International Business Machines Corporation
• Inventors: Mojsilovic, Aleksandra, Gomes, Jose, Rogowitz, Bernice
• Granted Patent: US 7,478,091 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06F 16/5862 using texture
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06V 10/40 Extraction of image or video features
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 30/274 Syntactic or semantic context, e.g. balancing
• CPC Class Codes: G16H 30/20 for handling medical images, e.g. DICOM, HL7 or PACS
• CPC Class Codes: G16H 30/40 for processing medical images, e.g. editing
• CPC Class Codes: G16H 50/20 for computer-aided diagnosis, e.g. based on medical expert systems
• CPC Class Codes: G16Z 99/00 Subject matter not provided for in other main groups of this subclass
• CPC Class Codes: Y10S 707/99936 Pattern matching access
• CPC Class Codes: Y10S 707/99945 Object-oriented database structure processing

Abstract

A method includes deriving a plurality of semantic categories for representing important semantic cues in images, where each semantic category is modeled through a combination of perceptual features that define the semantics of that category and that discriminate that category from other categories; for each semantic category, forming a set of the perceptual features comprising required features and frequently occurring features; comparing an image to said semantic categories; and classifying said image as belonging to one of said semantic categories if all of the required features and at least one of the frequently occurring features for that semantic category are present in said image. A database contains image information, where the image information includes at least one of already classified images, network locations of already classified images and documents containing already classified images. The database is searched for images matching an input query, comprising, e.g., an image, text, or both.

Claims

1-46. -46. (canceled)

47. An Internet searching tool, comprising:

an Internet searching robot comprising at least one autonomous searcher that locates images and returns at least one of image location data or image-related document location data, said searcher executing in a client computer;

a processor responsive to returned image location data and image-related document location data for accessing a located image for segmenting the located image and for extracting at least one feature from the located image for obtaining a set of image metadata;

storage for storing the set of image metadata; and

a user interface for entering a user query for use in searching the storage and for returning an image having image metadata that is relevant to the user query.

48. An Internet searching tool as in claim 47, where said user query comprises an image.

49. An Internet searching tool as in claim 47, where said user query comprises text.

50. An Internet searching tool as in claim 47, where there are a plurality of client computers and a corresponding plurality of searching robots individual ones of which execute in individual ones of said client computers.

51. An Internet searching tool as in claim 47, where said processor is responsive to the user query for segmenting an input query image and for extracting at least one feature from the input query image for obtaining a set of input image metadata, said processor using the input image metadata when searching the storage.

52. An Internet searching tool as in claim 47, where said processor derives a plurality of semantic categories for representing important semantic cues in images, where each semantic category is modeled through a combination of perceptual features that define the semantics of that category and that discriminate that category from other categories.

59. An Internet searching tool as in claim 52, where said perceptual features and combinations thereof are derived through subjective experiments performed with human observers.

60. An Internet searching tool as in claim 52, where:

said processor, for each semantic category, forms a set of the perceptual features comprising required features and frequently occurring features;

said processor compares said located image to said semantic categories; and

said processor classifies said located image as belonging to one of said semantic categories if all of the required features and at least one of the frequently occurring features for that semantic category are present in said located image.

61. A method as in claim 47, where said processor, when extracting said at least one feature, segments said located image by employing at least one of texture segmentation, color segmentation or foreground/background segmentation.

62. A method as in claim 61, where said processor when employing texture segmentation forms a texture map.

63. A method as in claim 62, where said processor when employing color segmentation forms a region map.

64. A method as in claim 63, where said processor when employing foreground/background segmentation further determines if there is at least one dominant object in the located image by using the texture map and the region map, and forms an object map.

65. A method as in claim 64, where the located image, texture map, region map and object map are further processed to compute, for each region in the region map and for each object in the object map, a set of local features comprising information descriptive of at least one of size, boundary, neighbors, boundary curvature, texture, mean color, color name, or shape properties for that region/object.

66. A method as in claim 65, where local features are analyzed to compute regional features for indicating a presence of semantic cues.

53. A method for searching the Internet, comprising:

providing at least one server accessible by a plurality of clients;

downloading a program to a client, said program comprising an Internet searching program operable for locating at least one type of data; and

executing said downloaded program on the client, and returning to said server Uniform Resource Locators (URLs) corresponding to located data.

54. A method as in claim 53, where the at least one type of data comprises image data.

55. A method as in claim 53, where said URLs comprise URLs of located digital images, and URLs of image related documents.

56. A system for obtaining information from the Internet, comprising at least one server accessible by a plurality of clients, said server operable for downloading a program to a client, said program comprising an Internet searching program operable for locating at least one type of data, said downloaded program being executable by said client and returning to said server Uniform Resource Locators (URLS) corresponding to located data, said server further comprising a database, searchable by the clients, for storing information corresponding to information returned by said downloaded program.

57. A system as in claim 56, where the at least one type of data comprises image data.

58. A system as in claim 56, where said URLs comprise URLs of located digital images, and URLs of image related documents.