System and method for enabling image searching using manual enrichment, classification, and/or segmentation

• US 8,315,442 B2
• Filed: 12/28/2009
• Issued: 11/20/2012
• Est. Priority Date: 05/09/2005
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Gokturk, Salih Burak, Sumengen, Baris, Vu, Diem, Dalal, Navneet, Yang, Danny, Lin, Xiaofan, Khan, Azhar, Shah, Munjal, Anguelov, Dragomir, Torresani, Lorenzo, Vanhoucke, Vincent
• Primary Examiner(s): STREGE, JOHN B
• Application Number:
  US12/648,275
• Publication Number:
  US 20100135597A1
• Time in Patent Office: 1,058 Days
• Field of Search: 382/118, 382/181, 382/173
• US Class Current: 382/118
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06F 16/5862 using texture
• CPC Class Codes: G06F 18/40 Software arrangements specially adapted for pattern recognition, e.g. user interfaces or toolboxes therefor
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection
• CPC Class Codes: G06V 10/40 Extraction of image or video features
• CPC Class Codes: G06V 10/945 User interactive design; Environments; Toolboxes

Abstract

Embodiments described herein provide for a system for creating a data collection of recognized images. The system includes an image analysis module that is configured to programmatically analyze individual images in a collection of images in order to determine information about each image in the collection. The system may also include a manual interface that is configured to (i) interface with one or more human editors, and (ii) displays a plurality of panels concurrently. Individual panels may be provided for one or more analyzed images, and individual panels may be configured to display information that is at least indicative of the one or more images of that panel and/or of the information determined from the one or more images. Additionally, the manual interface enables the one or more human editors to view the plurality of panels concurrently and to interact with each of the plurality of panels in order to correct or remove any information that is incorrectly determined from the image of that panel.

Claims

1. A computer-implemented method for processing images, the method being implemented by one or more processors and comprising:

receiving an image;

detecting a person as being present in the image;

determining information about the detected person, including at least one of a category or an identifier of the detected person;

identifying one or more assumptions that the person in the image is wearing one or more of a clothing or apparel; and

analyzing the image, using the one or more assumptions, to determine information about the clothing or apparel;

wherein analyzing the image includes segmenting the image to identify a set of objects other than the detected person, and analyzing a portion of the image that depicts the set of objects based on an assumption that the set of objects include the clothing or apparel worn by the person.

2. The method of claim 1, wherein determining information about the detected person includes analyzing at least one of text or metadata associated with the image.

3. The method of claim 1, wherein receiving the image includes retrieving the image from a webpage.

4. The method of claim 1, wherein determining information about the detected person includes extracting one or more visual features from the detected person in the image.

5. The method of claim 4, wherein the one or more visual features correspond to a global feature of the detected person.

6. The method of claim 4, wherein the one or more visual features correspond to a local feature.

7. The method of claim 4, wherein the one or more visual features correspond to a color, a texture, a pattern, or a shape of an object or region of the detected person.

8. The method of claim 1, wherein determining information about the detected person includes using metadata associated with the image.

9. The method of claim 1, wherein the category of the person is a celebrity.

10. The method of claim 1, wherein detecting the person includes detecting a face of the person.

11. A computer system comprising:

a memory that stores a set of instructions;

one or more processors that use the set of instructions to;

receive an image;

detect a person as being present in the image;

determine information about the detected person, including at least one of a category or an identifier of the detected person;

identify one or more assumptions that the person in the image is wearing one or more of a clothing or apparel; and

analyze the image, using the one or more assumptions, to determine information about the clothing or apparel;

wherein the image is analyzed by segmenting the image to identify a set of objects other than the detected person, and analyzing a portion of the image that depicts the set of objects based on an assumption that the set of objects include the clothing or apparel worn by the person.

12. The computer system of claim 11, wherein the category of the person is a celebrity.

13. The computer system of claim 11, wherein the one or more processors detect a face of the person.

14. A non-transitory computer-readable medium that stores instructions for processing images, the instruction being executable by one or more processors to cause the one or more processors to perform operations that comprise:

receiving an image;

detecting a person as being present in the image;

determining information about the detected person, including at least one of a category or an identifier of the detected person;

identifying one or more assumptions that the person in the image is wearing one or more of a clothing or apparel; and

analyzing the image, using the one or more assumptions, to determine information about the clothing or apparel;

wherein analyzing the image includes segmenting the image to identify a set of obiects other than the detected person, and analyzing a portion of the image that depicts the set of obiects based on an assumption that the set of objects include the clothing or apparel worn by the person.