Methods and apparatus for automated true object-based image analysis and retrieval

• US 8,908,997 B2
• Filed: 05/29/2014
• Issued: 12/09/2014
• Est. Priority Date: 05/05/2004
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Retterath, Jamie E., Laumeyer, Robert A.
• Primary Examiner(s): Carter, Aaron W
• Application Number:
  US14/290,155
• Publication Number:
  US 20140270508A1
• Time in Patent Office: 194 Days
• Field of Search: 382/165, 382/203, 382/305, 707/709, 707/769, 707/E17.03, 707/E17.108
• US Class Current: 382/305
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06F 16/5862 using texture
• CPC Class Codes: G06F 2218/12 Classification; Matching
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching

Abstract

The present invention is an automated and extensible system for the analysis and retrieval of images based on region-of-interest (ROI) analysis of one or more true objects depicted by an image. The system uses an ROI database that is a relational or analytical database containing searchable vectors that represent the images stored in a repository. Entries in the database are created by an image locator and ROI classifier that work to locate images within the repository and extract relevant information that will be stored in the ROI database. The ROI classifier analyzes objects in an image identify actual features of the true object. Graphical searches are performed by the collaborative workings of an image retrieval module, an image search requestor and an ROI query module. The image search requestor is an abstraction layer that translates user or agent search requests into the language understood by the ROI query.

Claims

1. A computerized system comprising:

an image locator configured to locate images for analysis within an image repository; and

a region of interest classifier that segments one or more images located by the image locator into a plurality of regions of interest, each region of interest containing image content associated with one or more actual features of a physical object in the region of interest, the region of interest classifier being configured to extract one or more physical descriptors from the image content for each region of interest as a physical feature vector, each physical descriptor corresponding to at least one of the one or more actual features of a given physical object in the region of interest associated with the image content.

2. The system of claim 1 wherein the physical feature vector identifies features belonging to a physical object feature space.

3. The system of claim 2, wherein the physical feature vector contains a description of the region of interest having a format indicated by a region of interest type.

4. The system of claim 2, wherein the physical feature vector comprises at least one of a physical color vector field, a physical shape vector field, or a physical texture vector field.

5. The system of claim 1, wherein the image locator is further configured to continually searching the repository for new images.

6. The system of claim 1, wherein the region of interest classifier is configured to segment the one or more images into regions of interest using one or more of segmentation, blob analysis, high frequency filter and Fourier transformation.

7. The system of claim 1, wherein the physical feature vector has a searchable data structure that comprises one or more of a physical color vector, a physical shape vector, and a physical texture vector.

8. The system of claim 7, wherein the physical color vector belongs to a physical object color space that describes a range of colors for the true objects that correspond to the images stored in the image repository.

9. The system of claim 7 wherein the physical color vector, the physical shape vector or the physical texture vector has a single-bit field.

10. The system of claim 7, wherein the physical color vector has multi-bit fields with quantum levels representing one or more of:

a probability that the physical object is a designated color;

a percentage of a designated color volume contained within a region of interest physical color representation;

a distance from a center of mass of the region of interest physical color representation to a center of mass of the designated color volume;

or

a measure signifying a relative strength that a representative color contributes to a physical color of an original object.

11. The system of claim 7, wherein the physical shape vector belongs to a physical object shape space that contains two dimensional representations of true objects that correspond to the images stored in the repository.

12. The system of claim 7, wherein the physical texture vector belongs to a physical object texture space that describes a range of textures of the true objects that correspond to the images stored in the repository.

13. The system of claim 7, wherein the physical shape vector has multi-bit fields with quantum levels representing one or more of:

a probability that the physical object is a designated shape;

a probability that the physical object contains feature aspects of the designated shape;

or

a measure signifying a relative strength that a representative shape to a physical shape of an original object.

14. The system of claim 7, wherein the physical texture vector has multi-bit fields with quantum levels representing one or more of:

a probability that the physical object is a designated texture;

a probability that the physical object is in a similar class of textures;

or

a measure signifying a relative strength that a representative texture contributes to a physical texture of an original object.

15. A computer-implemented method comprising:

locating, using one or more computers, images for analysis within an image repository;

segmenting, using the one or more computers, the located images into a plurality of regions of interest, each region of interest containing content associated with one or more actual features of a physical object; and

extracting, using the one or more computers, at least one physical descriptor from the image content for each region of interest as a physical feature vector, each physical descriptor corresponding to one or more of the actual features of a given physical object in the region of interest associated with the image content.

16. The method of claim 15, wherein the physical feature vector has a searchable data structure that comprises one or more of a physical color vector, a physical shape vector, and a physical texture vector.

17. The method of claim 16, wherein the physical color vector has multi-bit fields with quantum levels representing one or more of:

a probability that the physical object is a designated color;

a percentage of a designated color volume contained within a region of interest physical color representation;

a distance from a center of mass of the region of interest physical color representation to a center of mass of the designated color volume;

or

a measure signifying a relative strength that a representative color contributes to a physical color of an original object.

18. The method of claim 16, wherein the physical shape vector has multi-bit fields with quantum levels representing one or more of:

a probability that the physical object is a designated shape;

a probability that the physical object contains feature aspects of the designated shape;

or

a measure signifying a relative strength that a representative shape to a physical shape of an original object.

19. The method of claim 16, wherein the physical texture vector has multi-bit fields with quantum levels representing one or more of:

a probability that the physical object is a designated texture;

a probability that the physical object is in a similar class of textures;

or

a measure signifying a relative strength that a representative texture contributes to a physical texture of an original object.

20. A non-transitory computer readable recording medium recorded with a program that, when executed by one or more computers, causes the one or more computers to implement a method comprising:

locating images for analysis within an image repository;

segmenting the located images into a plurality of regions of interest, each region of interest containing content associated with one or more actual features of a physical object; and

extracting at least one physical descriptor from the image content for each region of interest as a physical feature vector, each physical descriptor corresponding to one or more of the actual features of a given physical object in the region of interest associated with the image content.