System and method for enabling search and retrieval from image files based on recognized information

• US 7,809,722 B2
• Filed: 10/07/2005
• Issued: 10/05/2010
• Est. Priority Date: 05/09/2005
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Like.Com (Alphabet Inc.)
• Inventors: Shah, Munjal, Lee, Kuang-Chih, Gokturk, Salih Burak, Vu, Diem, Anguelov, Dragomir, Vanhoucke, Vincent, Khan, Azhar, Yang, Danny
• Primary Examiner(s): Truong; Cam Y T
• Application Number:
  US11/246,741
• Publication Number:
  US 20060253491A1
• Time in Patent Office: 1,824 Days
• Field of Search: 382/305
• US Class Current: 707/736
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5846 using extracted text
• CPC Class Codes: G06F 16/5854 using shape and object relationship

Abstract

An embodiment provides for enabling retrieval of a collection of captured images that form at least a portion of a library of images. For each image in the collection, a captured image may be analyzed to recognize information from image data contained in the captured image, and an index may be generated, where the index data is based on the recognized information. Using the index, functionality such as search and retrieval is enabled. Various recognition techniques, including those that use the face, clothing, apparel, and combinations of characteristics may be utilized. Recognition may be performed on, among other things, persons and text carried on objects.

Claims

1. A computer-implemented method for enabling retrieval of a collection of images, the method being performed by a combination of hardware components that include one or more processors and memory resources, the method comprising:

programmatically analyzing each image of the images in at least a portion of the collection by (i) detecting presence of one or more objects in individual images that comprise the collection;

(ii) determining a type of each detected object; and

(iii) performing an image recognition process for said each detected object based on the determined type of said each detected object in order to determine image recognition information for said each detected object, the image recognition process being performed by computational steps that incorporate a set of parameters that are based on the type of the detected object, the image recognition process performed for one of the types of detected objects being different than the image recognition process performed for another one of the types of detected objects;

wherein the one or more detected objects in at least some of the analyzed images correspond to an item of clothing or apparel;

storing the image recognition information about the one or more detected objects that are detected from said programmatically analyzing each image, said image recognition information including a recognition signature for individually detected objects that correspond to the item of clothing or apparel, and said recognition information;

determining a criteria from an image input provided by a user;

wherein the determining a criteria from an image input includes determining a recognition signature for at least a portion of an image that is used in the image input; and

wherein the determining the recognition signature is performed for the item of clothing or apparel that appears in the portion of the image in the image input;

comparing the criteria to the stored image recognition information in order to identify one or more images in the collection that satisfy the criteria, wherein comparing the criteria to the stored image recognition information includes enabling the image recognition information for individual items of clothing or apparel to form at least a partial basis for comparison against the criteria of the image input in order to determine whether the image that contains the item of clothing or apparel satisfy the criteria of the image input;

wherein the comparing the criteria to the stored image recognition information further comprises computing a similarity between (i) features of the item of clothing or apparel that appears in the portion of the image in the image input and (ii) features of individual objects that appear in at least a portion of the images that comprise the collection; and

presenting the one or more images that satisfy the criteria to the user.

2. The method of claim 1, wherein analyzing each image includes generating a recognition signature for the detected one or more objects, including for the item of clothing or apparel, in individual images that comprise the collection.

3. The method of claim 1, wherein analyzing each image includes detecting the item of clothing as being worn by a person appearing in one or more of the analyzed images.

4. The method of claim 3, wherein analyzing each image includes identifying the person appearing in each of the one or more of the analyzed images based in part on a facial characteristic.

5. A computer-readable storage medium that stores instructions for performing steps comprising enabling searching of a collection of captured images that form at least a portion of a library of the images, wherein the instructions that when executed by one or more processors, cause the one or more processors to perform steps comprising:

programmatically analyzing each image of the images in at least a portion of the collection by (i) detecting presence of one or more objects in individual images that comprise the collection;

(ii) determining a type of each detected object; and

(iii) performing an image recognition process for said each detected object based on the determined type of said each detected object in order to determine image recognition information for said each detected object, the image recognition process being performed by computational steps that incorporate a set of parameters that are based on the type of the detected object, the image recognition process performed for one of the types of detected objects being different than the image recognition process performed for another one of the types of detected objects;

wherein the one or more detected objects in at least some of the analyzed images correspond to an item of clothing or apparel;

storing the image recognition information about the one or more detected objects that are detected from said programmatically analyzing each image, said image recognition information including a recognition signature for individually detected objects that correspond to the item of clothing or apparel, and said recognition information;

determining a criteria from an image input provided by a user;

wherein the determining a criteria from an image input includes determining a recognition signature for at least a portion of an image that is used in the image input; and

wherein the determining the recognition signature is performed for the item of clothing or apparel that appears in the portion of the image in the image input;

comparing the criteria to the stored image recognition information in order to identify one or more images in the collection that satisfy the criteria, wherein comparing the criteria to the stored image recognition information includes enabling the image recognition information for individual items of clothing or apparel to form at least a partial basis for comparison against the criteria of the image input in order to determine whether the image that contains the item of clothing or apparel satisfy the criteria of the image input;

wherein the comparing the criteria to the stored image recognition information further comprises computing a similarity between (i) features of the item of clothing or apparel that appears in the portion of the image in the image input and (ii) features of individual objects that appear in at least a portion of the images that comprise the collection; and

presenting the one or more images that satisfy the criteria to the user.