Methods and apparatus for automated true object-based image analysis and retrieval

• US 7,590,310 B2
• Filed: 05/05/2005
• Issued: 09/15/2009
• Est. Priority Date: 05/05/2004
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Facet Technology Corp.
• Inventors: Laumeyer, Robert A., Retterath, Jamie E.
• Primary Examiner(s): Ahmed; Samir A.
• Assistant Examiner(s): Koziol; Stephen R
• Application Number:
  US11/122,969
• Publication Number:
  US 20050271304A1
• Time in Patent Office: 1,594 Days
• Field of Search: 382/190, 382/305, 707/3
• US Class Current: 382/305
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06F 16/5862 using texture
• CPC Class Codes: G06F 2218/12 Classification; Matching
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching

Abstract

The present invention is an automated and extensible system for the analysis and retrieval of images based on region-of-interest (ROI) analysis of one or more true objects depicted by an image. The system uses a Regions Of Interest (ROI) database that is a relational or analytical database containing searchable vectors that represent the images stored in a repository. Entries in the ROI database are created by an image locator and ROI classifier that work in tandem to locate images within the repository and extract the relevant information that will be stored in the ROI database. Unlike existing region-of-interest search systems, the ROI classifier analyzes objects in an image to arrive at the actual features of the true object, instead of merely describing the features of the image of that object. Graphical searches are performed by the collaborative workings of an image retrieval module, an image search requestor and an ROI query module. The image search requestor is an abstraction layer that translates user or agent search requests into the language understood by the ROI query.

Claims

1. A computerized system for automated retrieval of images from an unstructured repository comprising:

a region of interest (ROI) database that contains searchable entries representing images stored in the repository;

an image locator that automatically locates images within the repository to populate the ROI database;

an ROI classifier that segments images located by the image locator into a plurality of ROIs wherein each ROI contains content associated with one or more actual features of a physical object, the ROI classifier having a plurality of computer-implemented physical descriptor extraction transform filters wherein each computer-implemented physical descriptor extraction transform filter is adapted to extract physical descriptors from the content for each ROI to be stored in the ROI database, each physical descriptor corresponding to one of the one or more actual features associated with the content and having a searchable data structure that includes at least an ROI type field and an ROI descriptor field wherein the searchable data structure is a physical color vector and wherein the physical color vector belongs to a physical object color space that describes a range of colors for the true objects that correspond to the images stored in the repository and wherein the physical color vector has a single-bit field and wherein the physical color vector has a multi-bit field representing an available quantum level selected from the set consisting of a probability that the object is a designated color with a designated color volume, a percentage of the designated color volume is contained within a ROI physical color representation, a distance from the center of mass of a ROI physical color representation to a center of mass of the designated color volume and a measure signifying the relative strength that a representative color contributes to a physical color of a original object;

an ROI query module that interrogates the ROI database by searching the ROI database for images containing content associated with at least one true object requested by a user search request;

an image search requestor that receives and formats the user search request for input to the ROI query module; and

an image retrieval module that utilizes the searchable entries in the ROI database to access the images stored in the repository that have content generally matching the true object requested by the user search request.

2. A computerized system for automated retrieval of images from an unstructured repository comprising:

a region of interest (ROI) database that contains searchable entries representing images stored in the repository;

an image locator that automatically locates images within the repository to populate the ROI database;

an ROI classifier that segments images located by the image locator into a plurality of ROIs wherein each ROI contains content associated with one or more actual features of a physical object, the ROI classifier having a plurality of computer-implemented physical descriptor extraction transform filters wherein each computer-implemented physical descriptor extraction transform filter is adapted to extract physical descriptors from the content for each ROI to be stored in the ROI database, each physical descriptor corresponding to one of the one or more actual features associated with the content and having a searchable data structure that includes at least an ROI type field and an ROI descriptor field wherein the searchable data structure is a physical shape vector and wherein the physical shape vector contains fields that represent two-dimensional representations of true objects, the system further including a shape vector lookup table containing projections that correspond to the fields within the physical shape vector;

an ROI query module that interrogates the ROI database by searching the ROI database for images containing content associated with at least one true object requested by a user search request;

an image search requestor that receives and formats the user search request for input to the ROI query module; and

an image retrieval module that utilizes the searchable entries in the ROI database to access the images stored in the repository that have content generally matching the true object requested by the user search request.

3. A computerized system for automated retrieval of images from an unstructured repository comprising:

a region of interest (ROI) database that contains searchable entries representing images stored in the repository;

an image locator that automatically locates images within the repository to populate the ROI database;

an ROI classifier that segments images located by the image locator into a plurality of ROIs wherein each ROI contains content associated with one or more actual features of a physical object, the ROI classifier having a plurality of computer-implemented physical descriptor extraction transform filters wherein each computer-implemented physical descriptor extraction transform filter is adapted to extract physical descriptors from the content for each ROI to be stored in the ROI database, each physical descriptor corresponding to one of the one or more actual features associated with the content and having a searchable data structure that includes at least an ROI type field and an ROI descriptor field wherein the searchable data structure is a physical shape vector and wherein the physical shape vector contains fields that represent two-dimensional representations of true objects, the system further including a shape vector lookup table containing projections that correspond to the fields within the physical shape vector and wherein the physical shape vector has a multi-bit field representing an available quantum level selected from the group consisting of a probability that the true object is a designated shape, that the true object contains feature aspects of the designated shape, and that a measure that a representative shape contributes to a physical shape of the true object;

an ROI query module that interrogates the ROI database by searching the ROI database for images containing content associated with at least one true object requested by a user search request;

an image search requestor that receives and formats the user search request for input to the ROI query module; and

an image retrieval module that utilizes the searchable entries in the ROI database to access the images stored in the repository that have content generally matching the true object requested by the user search request.

4. A computerized system for automated retrieval of images from an unstructured repository comprising:

a region of interest (ROI) database that contains searchable entries representing images stored in the repository;

an image locator that automatically locates images within the repository to populate the ROI database;

an ROI classifier that segments images located by the image locator into a plurality of ROIs wherein each ROI contains content associated with one or more actual features of a physical object, the ROI classifier having a plurality of computer-implemented physical descriptor extraction transform filters wherein each computer-implemented physical descriptor extraction transform filter is adapted to extract physical descriptors from the content for each ROI to be stored in the ROI database, each physical descriptor corresponding to one of the one or more actual features associated with the content and having a searchable data structure that includes at least an ROI type field and an ROI descriptor field wherein the searchable data structure is a physical texture vector and wherein the physical texture vector belongs to a physical object texture space that describes a range of textures of the true objects that correspond to the images stored in the repository and wherein the physical texture vector has a single-bit field and wherein the physical texture vector has a multi-bit field representing an available quantum level selected from the group consisting of a probability that the true object is a designated texture, that the true object is in a similar class of textures, and that a measure that a representative texture contributes to a physical texture of the true object;

an ROI query module that interrogates the ROI database by searching the ROI database for images containing content associated with at least one true object requested by a user search request;

an image search requestor that receives and formats the user search request for input to the ROI query module; and

an image retrieval module that utilizes the searchable entries in the ROI database to access the images stored in the repository that have content generally matching the true object requested by the user search request.

5. A computerized system for automated retrieval of images from an unstructured repository comprising:

a region of interest (ROI) database that contains searchable entries representing images stored in the repository;

an image locator that automatically locates images within the repository to populate the ROI database;

an ROI classifier that segments images located by the image locator into a plurality of ROIs wherein each ROI contains content associated with one or more actual features of a physical object, the ROI classifier having a plurality of computer-implemented physical descriptor extraction transform filters wherein each computer-implemented physical descriptor extraction transform filter is adapted to extract physical descriptors from the content for each ROI to be stored in the ROI database, each physical descriptor corresponding to one of the one or more actual features associated with the content and having a searchable data structure that includes at least an ROI type field and an ROI descriptor field wherein the searchable data structure is a physical texture vector and wherein the physical texture vector belongs to a physical object texture space that describes a range of textures of the true objects that correspond to the images stored in the repository and wherein the physical texture vector has a single-bit field and wherein the physical texture vector has a multi-bit field representing an available quantum level selected from the group consisting of a probability that the true object is a designated texture, that the true object is in a similar class of textures, and that a measure that a representative texture contributes to a physical texture of the true object and wherein the physical texture vector further includes fields with physical descriptors selected from the group consisting of shiny, dull, metallic, transparent, translucent, and reflective;

an ROI query module that interrogates the ROI database by searching the ROI database for images containing content associated with at least one true object requested by a user search request;

an image search requestor that receives and formats the user search request for input to the ROI query module; and

an image retrieval module that utilizes the searchable entries in the ROI database to access the images stored in the repository that have content generally matching the true object requested by the user search request.