System and method for providing objectified image renderings using recognition information from images

• US 8,139,900 B2
• Filed: 06/21/2010
• Issued: 03/20/2012
• Est. Priority Date: 05/09/2005
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Like.Com (Alphabet Inc.)
• Inventors: Gokturk, Salih Burak, Anguelov, Dragomir, Vanhoucke, Vincent, Lee, Kuang-Chih, Vu, Diem, Yang, Danny, Shah, Munjal, Khan, Azhar
• Primary Examiner(s): Kassa, Yosef
• Application Number:
  US12/819,970
• Publication Number:
  US 20110026853A1
• Time in Patent Office: 638 Days
• Field of Search: 382/112, 382/115, 382/118, 382/209, 382/278, 382/282, 382/305
• US Class Current: 382/305
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5846 using extracted text
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 18/23 Clustering techniques
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 20/70 Labelling scene content, e.g. deriving syntactic or semantic representations
• CPC Class Codes: G06V 40/107 Static hand or arm
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/172 Classification, e.g. identification

Abstract

An embodiment provides for enabling retrieval of a collection of captured images that form at least a portion of a library of images. For each image in the collection, a captured image may be analyzed to recognize information from image data contained in the captured image, and an index may be generated, where the index data is based on the recognized information. Using the index, functionality such as search and retrieval is enabled. Various recognition techniques, including those that use the face, clothing, apparel, and combinations of characteristics may be utilized. Recognition may be performed on, among other things, persons and text carried on objects.

Claims

1. A computer-implemented method comprising:

supplementing an image with metadata that identifies an object in the image;

providing the metadata with the image when the image is rendered so that at least a portion of the image depicting the object is interactive, in order to enable a user to enter a selection input that corresponds to the object; and

performing an action in response to detecting the selection input that corresponds to the object depicted in the image.

2. The method of claim 1, wherein performing an action includes specifying a content for display in response to detecting the selection input.

3. The method of claim 2, wherein the content includes information that is based on the metadata.

4. The method of claim 1, wherein performing an action includes performing a search of other images using a search criteria that specifies the object of the selection input.

5. The method of claim 4, wherein the search criteria is based on text data associated with the object of the selection input.

6. The method of claim 4, wherein the search criteria is based on recognition information determined from the object of the selection input.

7. The method of claim 1, further comprising storing the metadata in a header of a file of the image.

8. The method of claim 1, wherein supplementing the image with the metadata includes providing the metadata in a separate file in association with a file of the image.

9. The method of claim 2, wherein specifying the content for display includes displaying additional information about the object that corresponds to the selection input, wherein the additional information is identified from the metadata.

10. The method of claim 2, wherein specifying the content for display includes identifying additional images that contain the selected one or more objects.

11. A computer storage medium encoding computer executable instructions that when executed by one or more processors, cause the one or more processors to perform operations comprising:

supplementing an image with metadata that identifies an object in the image;

providing the metadata with the image when the image is rendered so that at least a portion of the image depicting the object is interactive, in order to enable a user to enter a selection input that corresponds to the object; and

performing an action in response to detecting the selection input that corresponds to the object depicted in the image.

12. The computer storage medium of claim 11, wherein the operation for causing the image to be rendered includes specifying a content that is to be displayed in response to detecting the selection input.

13. The computer storage medium of claim 12, wherein the content includes information that is based on the metadata.

14. The computer storage medium of claim 11, further comprising instructions for performing a search of other images using a search criteria that specifies the object of the selection input.

15. The computer storage medium of claim 14, wherein the search criteria is based on text data associated with the object of the selection input.

16. The computer storage medium of claim 14, wherein the search criteria is based on recognition information determined from the object of the selection input.

17. The computer storage medium of claim 11, further comprising instructions for storing the metadata in a header of a file of the image.

18. The computer storage medium of claim 11, wherein the operation for supplementing the stored data includes providing the metadata in a separate file that is associated with a file of the image.

19. The computer storage medium of claim 12, wherein the operation for displaying the content includes displaying additional information about the object that corresponds to the selection input, wherein the additional information is identified from the metadata.

20. The computer storage medium of claim 12, wherein the operation for displaying the content in response to detecting the selection input includes identifying additional images that contain the selected one or more objects.

29. The system of claim 12, wherein the one or more processors are configured to specify the content to display, in response to detecting the selection input, by identifying additional information to be displayed about the object that corresponds to the selection input, wherein the additional information is identified from the metadata.

30. The computer storage medium of claim 12, wherein the one or more processors are configured to specify the content to display, in response to detecting the selection input, by identifying additional images that contain the selected one or more objects.

21. A system comprising:

one or more processors; and

a memory coupled to the one or more processors, wherein the memory stores instructions that, when executed by the one or more processors, cause the one or more processors to;

supplement an image with metadata that identifies an object in the image;

providing the metadata with the image when the image is rendered so that at least a portion of the image depicting the object is interactive, in order to enable a user to enter a selection input that corresponds to the object; and

perform an action in response to detecting the selection input that corresponds to the object depicted in the image.

22. The system of claim 21, wherein the one or more processors are configured to specify a content that is to be displayed in response to detecting the selection input.

23. The system of claim 22, wherein the content includes information that is based on the metadata.

24. The system of claim 21, wherein the one or more processors are configured to perform a search of other images using a search criteria that specifies the object of the selection input.

25. The system of claim 24, wherein the search criteria is based on text data associated with the object of the selection input.

26. The system of claim 24, wherein the search criteria is based on recognition information determined from the object of the selection input.

27. The system of claim 21, wherein the one or more processors store the metadata in a header of a file of the image.

28. The system of claim 21, wherein the one or more processors are configured to supplement the stored data by providing the metadata in a separate file in association with a file of the image.