System and method for enabling the use of captured images through recognition

• US 8,897,505 B2
• Filed: 09/13/2012
• Issued: 11/25/2014
• Est. Priority Date: 05/09/2005
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Gokturk, Salih Burak, Anguelov, Dragomir, Vanhoucke, Vincent, Lee, Kuang-Chih, Vu, Diem, Yang, Danny, Shah, Munjal, Khan, Azhar
• Primary Examiner(s): STREGE, JOHN B
• Application Number:
  US13/615,276
• Publication Number:
  US 20130121589A1
• Time in Patent Office: 803 Days
• Field of Search: 382/118, 382/181
• US Class Current: 382/118
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06F 16/53 Querying
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/5846 using extracted text
• CPC Class Codes: G06F 16/587 using geographical or spatial information, e.g. location
• CPC Class Codes: G06V 40/107 Static hand or arm

Abstract

An embodiment provides for enabling retrieval of a collection of captured images that form at least a portion of a library of images. For each image in the collection, a captured image may be analyzed to recognize information from image data contained in the captured image, and an index may be generated, where the index data is based on the recognized information. Using the index, functionality such as search and retrieval is enabled. Various recognition techniques, including those that use the face, clothing, apparel, and combinations of characteristics may be utilized. Recognition may be performed on, among other things, persons and text carried on objects.

Claims

1. A method performed by at least one processor, the method comprising:

analyzing a first captured image and a second captured image, including;

detecting a first face in the first captured image and a second face in the second captured image; and

detecting a first clothing item associated with the first face and a second clothing item associated with the second face; and

determining that the first captured image and the second captured image belong to a common cluster of images;

in response to determining that the first captured image and the second captured image belong to a common cluster of images, determining an identity assignment for the first and second faces based on the first and second faces and the first and second clothing items.

2. The method of claim 1, further comprising adding the identity assignment to an index and enabling the index to be used for searching images.

3. The method of claim 2, wherein enabling the index to be searched includes enabling the index to be searched using input that corresponds to a portion of an image.

4. The method of claim 1, wherein detecting the first face comprises performing normalization on the first face, including normalizing effects of lighting when the image was captured using at least one of a histogram or linear ramp intensity analysis.

5. The method of claim 4, wherein analyzing the captured image includes:

generating a recognition signature for a person associated with the identity assignment that identifies the person from other persons recognized in a collection of images based at least in part on the face of the person.

6. The method of claim 5, wherein analyzing the captured image includes:

detecting a time when the image was captured, and

wherein generating the recognition signature includes generating the recognition signature based at least in part on the detected time.

7. The method of claim 1, wherein determining that the first captured image and the second captured image belong to a common cluster of images comprises determining that the first captured image and the second captured image were captured during a same event.

8. The method of claim 7, wherein determining that the first captured image and the second captured image were captured during a same event comprises using time information and location information for the first and second captured images.

9. The method of claim 1, wherein determining the identity assignment comprises:

determining a clothing vector difference using the first clothing information and the second clothing information; and

determining a face vector difference using the first face and the second face.

10. The method of claim 9, wherein determining the identity assignment comprises determining a final difference vector as a weighted combination of the clothing vector difference and the face vector difference.

11. A system comprising one or more processors configured to perform operations comprising:

analyzing a first captured image and a second captured image, including;

detecting a first face in the first captured image and a second face in the second captured image; and

detecting a first clothing item associated with the first face and a second clothing item associated with the second face; and

determining that the first captured image and the second captured image belong to a common cluster of images;

in response to determining that the first captured image and the second captured image belong to a common cluster of images, determining an identity assignment for the first and second faces based on the first and second faces and the first and second clothing items.

12. The system of claim 11, the operations further comprising adding the identity assignment to an index and enabling the index to be used for searching images.

13. The system of claim 12, wherein enabling the index to be searched includes enabling the index to be searched using input that corresponds to a portion of an image.

14. The system of claim 11, wherein detecting the first face comprises performing normalization on the first face, including normalizing effects of lighting when the image was captured using at least one of a histogram or linear ramp intensity analysis.

15. The system of claim 14, wherein analyzing the captured image includes:

generating a recognition signature for a person associated with the identity assignment that identifies the person from other persons recognized in a collection of images based at least in part on the face of the person.

16. The system of claim 14, wherein analyzing the captured image includes:

detecting a time when the image was captured, and

wherein generating the recognition signature includes generating the recognition signature based at least in part on the detected time.

17. The system of claim 11, wherein determining that the first captured image and the second captured image belong to a common cluster of images comprises determining that the first captured image and the second captured image were captured during a same event.

18. The system of claim 17, wherein determining that the first captured image and the second captured image were captured during a same event comprises using time information and location information for the first and second captured images.

19. The system of claim 11, wherein determining the identity assignment comprises:

determining a clothing vector difference using the first clothing information and the second clothing information; and

determining a face vector difference using the first face and the second face.

20. The system of claim 11, wherein determining the identity assignment comprises determining a final difference vector as a weighted combination of the clothing vector difference and the face vector difference.