System and method for providing objectified image renderings using recognition information from images

• US 8,630,513 B2
• Filed: 02/10/2012
• Issued: 01/14/2014
• Est. Priority Date: 05/09/2005
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Gokturk, Salih Burak, Anguelov, Dragomir, Vanhoucke, Vincent, Lee, Kuang-Chih, Vu, Diem, Yang, Danny, Shah, Munjal, Khan, Azhar
• Primary Examiner(s): Kassa, Yosef
• Application Number:
  US13/371,196
• Publication Number:
  US 20120304125A1
• Time in Patent Office: 704 Days
• Field of Search: 382/115, 382/117, 382/118, 382/278, 382/305, 340/5.81, 340/5.82, 340/5.83, 902/3, 902/5
• US Class Current: 382/305
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5846 using extracted text
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 18/23 Clustering techniques
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 20/70 Labelling scene content, e.g. deriving syntactic or semantic representations
• CPC Class Codes: G06V 40/107 Static hand or arm
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/172 Classification, e.g. identification

Abstract

An embodiment provides for enabling retrieval of a collection of captured images that form at least a portion of a library of images. For each image in the collection, a captured image may be analyzed to recognize information from image data contained in the captured image, and an index may be generated, where the index data is based on the recognized information. Using the index, functionality such as search and retrieval is enabled. Various recognition techniques, including those that use the face, clothing, apparel, and combinations of characteristics may be utilized. Recognition may be performed on, among other things, persons and text carried on objects.

Claims

1. A computer-implemented method for displaying images to a user, the method comprising:

supplementing an image with object data that identifies an object in the image;

extrapolating information about the object depicted in the image based on the object data;

storing the extrapolated information in a header of a file of the image; and

providing the extrapolated information with the image as renderable image data.

2. The method of claim 1, further comprising:

enabling at least a portion of the image to be interactive; and

performing an action using the extrapolated information in response to detecting a user interaction with the image.

3. The method of claim 2, wherein performing an action includes performing a search of other images using a search criteria that is based on the extrapolated information.

4. The method of claim 3, wherein extrapolating information about the object comprises extrapolating the information based on text data associated with the identified object.

5. The method of claim 3, wherein extrapolating information about the object comprises extrapolating the information based on metadata associated with the identified object.

6. The method of claim 2, wherein the user interaction includes a selection input that corresponds to the object depicted in the image.

7. The method of claim 6, wherein performing an action includes identifying other captured images that contain the selected object.

8. A computer storage medium encoding computer executable instructions that when executed by one or more processors, cause the one or more processors to perform operations comprising:

supplementing an image with object data that identifies an object in the image;

extrapolating information about the object depicted in the image based on the object data;

storing the extrapolated information in a header of a file of the image; and

providing the extrapolated information with the image as renderable image data.

9. The computer storage medium of claim 8, further comprising instructions for:

enabling at least a portion of the image to be interactive; and

performing an action using the extrapolated information in response to detecting a user interaction with the image.

10. The computer storage medium of claim 9, wherein the operation for performing an action includes performing a search of other images using a search criteria that is based on the extrapolated information.

11. The computer storage medium of claim 10, wherein the operation for extrapolating information about the object includes extrapolating the information based on text data associated with the identified object.

12. The computer storage medium of claim 10, wherein the operation for extrapolating information about the object includes extrapolating the information based on metadata associated with the identified object.

13. The computer storage medium of claim 9, wherein the user interaction includes a selection input that corresponds to the object depicted in the image.

14. A system comprising:

one or more processors; and

a memory coupled to the one or more processors, wherein the memory stores instructions that, when executed by the one or more processors, cause the one or more processors to;

supplement an image with object data that identifies an object in the image;

extrapolate information about the object depicted in the image based on the object data;

store the extrapolated information in a header of a file of the image; and

provide the extrapolated information with the image as renderable image data.

15. A computer-implemented method for displaying images to a user, the method comprising:

supplementing an image with object data that identifies an object in the image;

extrapolating information about the object depicted in the image based on the object data; and

providing the extrapolated information in a separate file in association with a file of the image;

wherein the extrapolated information is provided as renderable image data.

16. The method of claim 15, further comprising:

enabling at least a portion of the image to be interactive; and

performing an action using the extrapolated information in response to detecting a user interaction with the image.

17. The method of claim 16, wherein performing an action includes performing a search of other images using a search criteria that is based on the extrapolated information.

18. The method of claim 17, wherein extrapolating information about the object comprises extrapolating the information based on text data associated with the identified object.

19. The method of claim 17, wherein extrapolating information about the object comprises extrapolating the information based on metadata associated with the identified object.

20. The method of claim 16, wherein the user interaction includes a selection input that corresponds to the object depicted in the image.

21. The method of claim 20, wherein performing an action includes identifying other captured images that contain the selected object.