System and method for providing objectified image renderings using recognition information from images

• US 7,783,135 B2
• Filed: 10/07/2005
• Issued: 08/24/2010
• Est. Priority Date: 05/09/2005
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Like.Com (Alphabet Inc.)
• Inventors: Shah, Munjal, Lee, Kuang-Chih, Gokturk, Salih Burak, Vu, Diem, Anguelov, Dragomir, Vanhoucke, Vincent, Khan, Azhar, Yang, Danny
• Primary Examiner(s): Kassa; Yosef
• Application Number:
  US11/246,434
• Publication Number:
  US 20060251338A1
• Time in Patent Office: 1,782 Days
• Field of Search: 382/118, 382/128, 382/209, 382/219, 382/305
• US Class Current: 382/305
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5846 using extracted text
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 18/23 Clustering techniques
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 20/70 Labelling scene content, e.g. deriving syntactic or semantic representations
• CPC Class Codes: G06V 40/107 Static hand or arm
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/172 Classification, e.g. identification

Abstract

An embodiment provides for enabling retrieval of a collection of captured images that form at least a portion of a library of images. For each image in the collection, a captured image may be analyzed to recognize information from image data contained in the captured image, and an index may be generated, where the index data is based on the recognized information. Using the index, functionality such as search and retrieval is enabled. Various recognition techniques, including those that use the face, clothing, apparel, and combinations of characteristics may be utilized. Recognition may be performed on, among other things, persons and text carried on objects.

Claims

1. A method for displaying images to a user, the method being implemented by one or more processors that perform steps comprising:

using one or more processors to perform steps comprising;

performing image analysis on a captured image to detect and identify one or more objects in the captured image, wherein performing the image analysis includes performing a recognition or similarity process that is selected or configured for a type of each of the one or more detected objects;

associating stored data corresponding to the captured image with information that identifies the one or more objects;

presenting the captured image using the stored data and the information in order to identify, and make selectable the one or more objects from the remainder of the captured image;

detecting a selection of one of the objects by the user when the captured image is presented; and

performing an action in response to detecting the selection of the object.

2. The method of claim 1, wherein performing an action includes displaying a content in response to detecting the selection.

3. The method of claim 2, wherein displaying the content includes selectively displaying the information that identifies the one or more objects.

4. The method of claim 2, wherein displaying a content in response to detecting the selection includes displaying additional information about the selected object, wherein the additional information is identified from the information that identifies the one or more objects.

5. The method of claim 2, wherein performing the action includes identifying other captured images that contain the same selected object.

6. The method of claim 2, wherein performing the action includes identifying other captured images that contain one or more objects that are determined to be relevant to the selected object.

7. The method of claim 1, wherein performing an action includes performing a search of other images using at least one search criterion corresponding to the given one of the one or more objects.

8. The method of claim 7, wherein the at least one search criterion is based on text data associated with the given one of the one or more objects.

9. The method of claim 7, wherein the at least one search criterion is based on recognition information determined from the given one of the one or more objects.

10. The method of claim 1, further comprising storing the data in a header of a file of the captured image.

11. The method of claim 1, wherein in detecting a selection of a given one of the one or more objects, the method further comprises determining a recognition signature of the selected object, and wherein performing the action that includes performing a search of a collection of images for any image that contains an object with a same or similar recognition signature.

12. A method for displaying images to a user, the method being implemented by one or more processors that perform steps comprising:

using one or more processors to perform steps comprising;

designating a kind of object to be detected from recognition analysis performed on captured images;

detecting one or more objects on a captured image, wherein the one or more objects are of the kind that are designated to be detected;

supplementing stored data that corresponds to the captured image with information that identifies each detected object; and

using the stored data and the information that identifies each detected object to present the captured image with the detected objects wherein the detected objects are presented as being separately selectable;

detecting input corresponding to selection of one of the separately selectable objects apart from a remainder of the captured image; and

responsive to detecting the input, displaying at least one of information or content that is selected based on the detected object.

13. The method of claim 12, wherein detecting one or more objects includes detecting at least a portion of a person in the image.

14. The method of claim 13, wherein displaying at least one of information or content includes displaying other captured images with the same detected person.

15. The method of claim 13, wherein detecting at least a portion of the person includes recognizing the person, and wherein displaying at least one of information or content that is selected based on the detected object includes displaying at least one of other images that are determined to have the same person.

16. The method of claim 12, wherein the object includes a face of the person.

17. The method of claim 12, wherein detecting one or more objects includes detecting an object carrying a text in the image.

18. The method of claim 17, further comprising recognizing the text, and wherein displaying at least one of information or content includes identifying one or more content or information that relate to the recognized text.

19. The method of claim 18, wherein the recognized text corresponds to a geographical identifier, and wherein displaying at least one of information or content includes displaying resources that include at least one of information and content about the geographical identifier.

20. The method of claim 18, wherein the recognized text corresponds to a business establishment, and wherein displaying at least one of information or content includes displaying resources that include at least one of information and content about the business establishment.

21. The method of claim 18, wherein identifying one or more content or information that relate to the recognized text includes identifying one or more web sites that are pertinent to the recognized text.

22. The method of claim 21, wherein identifying one or more web sites includes programmatically submitting the recognized text to a search engine.

23. An image viewing system comprising:

an image analyzer configured to (i) detect and identify one or more objects in a captured image and (ii) to associate stored data corresponding to the captured image with information that identifies the one or more objects;

a memory that stores (I) an image file of the data corresponding to the captured image, and (ii) the information that identifies the one or more objects; and

a user-interface configured (i) to present the captured image, (ii) to identify and make selectable the one or more objects from the remainder of the image, and (iii) to perform an action in response to detecting the selection of one of the objects.

24. The image viewing system of claim 23, wherein the information that identifies the one or more objects in the image file is stored in a header of the image file.

25. The image viewing system of claim 23, wherein the information that identifies the one or more objects in the image file is stored external to the image file.

26. The image viewing system of claim 23, wherein the user-interface is configured to present an image using data from the image file, wherein the user-interface presents the one or more objects as interactive features of the image.

27. The image viewing system of claim 26, wherein the user-interface is configured to display information corresponding to the information that identifies one or more objects in the image file in response to a user action.

28. The image viewing system of claim 26, wherein the user-interface is configured to initiate a search for other images based on a user action.