System and method for enabling the use of captured images through recognition

• US 8,649,572 B2
• Filed: 02/27/2009
• Issued: 02/11/2014
• Est. Priority Date: 05/09/2005
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Gokturk, Salih Burak, Anguelov, Dragomir, Vanhoucke, Vincent, Lee, Kuang-Chih, Vu, Diem, Yang, Danny, Shah, Munjal, Khan, Azhar
• Primary Examiner(s): STREGE, JOHN B
• Application Number:
  US12/395,422
• Publication Number:
  US 20090196510A1
• Time in Patent Office: 1,810 Days
• Field of Search: 382/118, 382/181
• US Class Current: 382/118
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06F 16/53 Querying
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/5846 using extracted text
• CPC Class Codes: G06F 16/587 using geographical or spatial information, e.g. location
• CPC Class Codes: G06V 40/107 Static hand or arm

Abstract

An embodiment provides for enabling retrieval of a collection of captured images that form at least a portion of a library of images. For each image in the collection, a captured image may be analyzed to recognize information from image data contained in the captured image, and an index may be generated, where the index data is based on the recognized information. Using the index, functionality such as search and retrieval is enabled. Various recognition techniques, including those that use the face, clothing, apparel, and combinations of characteristics may be utilized. Recognition may be performed on, among other things, persons and text carried on objects.

Claims

1. A computer-implemented method comprising:

analyzing an image to detect a portion of the image that corresponds to a person, and to recognize information from the image for the person;

determining a set of metadata for the image, wherein the set of metadata includes the recognized information, and data that identifies the portion of the image apart from a remainder of the image;

associating a set of permissions with the set of metadata;

providing the set of metadata with the image in accordance with the set of permissions;

wherein providing the set of metadata includes;

enabling the portion of the image depicting the person to be selectable separate from the remainder of the image, and providing the recognized information with the image of the person in response to a user selecting the portion of the image that corresponds to the person.

2. The method of claim 1, further comprising:

generating index data for an index, the index data being based on the recognized information;

in response to the user selecting the portion of the image, enabling a search operation to be performed using the index.

3. The method of claim 1, wherein analyzing the image includes determining an identifier of the person depicted in the image based on the recognized information.

4. The method of claim 1, wherein analyzing the image includes recognizing information for the person corresponding to one or more of:

(i) an apparel of the person, (ii) a clothing item of the person, or (iii) a time when the image was captured.

5. The method of claim 1, wherein analyzing the image further comprises detecting a text image contained in the image.

6. The method of claim 5, wherein analyzing the image includes recognizing text from the text image.

7. The method of claim 6, further comprising identifying one or more text items by extrapolating the recognized text.

8. The method of claim 7, wherein identifying one or more text items by extrapolating the recognized text includes identifying the one or more text items based on one or more of:

(i) a geographic relationship, (ii) a dictionary, and (iii) a thesaurus.

9. The method of claim 6, wherein analyzing the image includes determining a relevance of the text image based on other information determined from the image.

10. The method of claim 1, further comprising providing the image for display in a social network.

11. The method of claim 1, further comprising enabling, using the set of metadata, the portion of the analyzed image to be used as a search criterion in response to the user selecting the portion of the image.

12. The method of claim 11, further comprising using an index associated with a social network to identify one or more images that can be correlated to the search criterion.

13. The method of claim 1, wherein the recognized information corresponds to a name, and wherein providing the image for display includes displaying the name of the person as an overlay of the image.

14. A system comprising:

one or more processors;

a memory resource that stores instructions for execution by the one or more processors;

wherein the one or more processors are configured to execute instructions stored in the memory resource in performing operations comprising;

analyze an image to detect a portion of the image that corresponds to a person, and to recognize information from the image for the person;

determine a set of metadata for the image, wherein the set of metadata includes the recognized information, and data that identifies the portion of the image apart from a remainder of the image;

associate a set of permissions with the set of metadata;

provide the set of metadata with the image in accordance with the set of permissions;

wherein the set of metadata is provided by;

enabling the portion of the image depicting the person to be selectable separate from the remainder of the image, and

providing the recognized information with the image of the person in response to a user selecting the portion of the image that corresponds to the person.

15. The system of claim 14, further comprising:

generating index data for an index, the index data being based on the recognized information;

in response to the user selecting the portion of the image, enabling a search operation to be performed using the index.

16. The system of claim 14, wherein analyzing the image includes determining an identifier of the person depicted in the image based on the recognized information.

17. The system of claim 14, further comprising providing the image for display in a social network.

18. The system of claim 14, further comprising enabling, using the set of metadata, the portion of the analyzed image to be used as a search criterion in response to the user selecting the portion of the image.

19. The system of claim 18, further comprising using an index associated with a social network to identify one or more images that can be correlated to the search criterion.

20. The system of claim 14, wherein the recognized information corresponds to a name, and wherein providing the image for display includes displaying the name of the person as an overlay of the image.

21. A non-transitory computer readable medium that stores instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:

analyzing an image to detect a portion of the image that corresponds to a person, and to recognize information from the image for the person;

determining a set of metadata for the image, wherein the set of metadata includes the recognized information, and data that identifies the portion of the image apart from a remainder of the image;

associating a set of permissions with the set of metadata;

providing the set of metadata with the image in accordance with the set of permissions;

wherein providing the set of metadata includes;

enabling the portion of the image depicting the person to be

selectable separate from the remainder of the image, and

providing the recognized information with the image of the person in response to a user selecting the portion of the image that corresponds to the person.

22. The computer-readable medium of claim 21, further comprising instructions for:

generating index data for an index, the index data being based on the recognized information; and

in response to the user selecting the portion of the image, enabling a search operation to be performed using the index.

23. The computer-readable medium of claim 21, wherein the instructions for analyzing the image include instructions for determining an identifier of the person depicted in the image based on the recognized information.

24. The computer-readable medium of claim 21, wherein the instructions for analyzing the image include instructions for recognizing information for the person corresponding to one or more of:

(i) an apparel of the person, (ii) a clothing item of the person, or (iii) a time when the image was captured.

25. The computer-readable medium of claim 21, wherein the instructions for analyzing the image include instructions for detecting a text image contained in the image.

26. The computer-readable medium of claim 25, wherein the instructions for analyzing the image include instructions for recognizing text from the text image.

27. The computer-readable medium of claim 26, further comprising instructions for identifying one or more text items by extrapolating the recognized text.

28. The computer-readable medium of claim 27, wherein the instructions for identifying one or more text items by extrapolating the recognized text include instructions for identifying the one or more text items based on one or more of:

(i) a geographic relationship, (ii) a dictionary, and (iii) a thesaurus.

29. The computer-readable medium of claim 26, wherein the instructions for analyzing the image include instructions for determining a relevance of the text image based on other information determined from the image.

30. The computer-readable medium of claim 21, further comprising instructions for providing the image for display in a social network.

31. The computer-readable medium of claim 21, further comprising instructions for enabling, using the set of metadata, the portion of the analyzed image to be used as a search criterion in response to the user selecting the portion of the image.

32. The computer-readable medium of claim 26, further comprising instructions for using an index associated with a social network to identify one or more images that can be correlated to the search criterion.

33. The computer-readable medium of claim 21, wherein the recognized information corresponds to a name, and wherein providing the image for display includes displaying the name of the person as an overlay of the image.