System and method for enabling search and retrieval from image files based on recognized information

• US 20060253491A1
• Filed: 10/07/2005
• Published: 11/09/2006
• Est. Priority Date: 05/09/2005
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Like.Com (Alphabet Inc.)
• Inventors: Lee, Kuang-chih, Shah, Munjal, Gokturk, Salih Burak, Vu, Diem, Anguelov, Dragomir, Vanhoucke, Vincent, Khan, Azhar, Yang, Danny
• Granted Patent: US 7,809,722 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5846 using extracted text
• CPC Class Codes: G06F 16/5854 using shape and object relationship

Abstract

An embodiment provides for enabling retrieval of a collection of captured images that form at least a portion of a library of images. For each image in the collection, a captured image may be analyzed to recognize information from image data contained in the captured image, and an index may be generated, where the index data is based on the recognized information. Using the index, functionality such as search and retrieval is enabled. Various recognition techniques, including those that use the face, clothing, apparel, and combinations of characteristics may be utilized. Recognition may be performed on, among other things, persons and text carried on objects.

Claims

1. A method for enabling retrieval of a collection of captured images that form at least a portion of a library of images, the method comprising:

for each image in the collection, performing steps comprising;

analyzing image data of the image to determine one or more information items from one or more detected objects in the captured image;

associating the one or more information items with the captured image;

in response to receiving an input from a user, performing steps comprising;

correlating the input to at least a first information item;

retrieving at least one of one or more images in the collection that have been determined to have the at least the first information item.

2. The method of claim 1, wherein analyzing image data includes generating a recognition signature for one or more persons in one or more images in the collection.

3. The method of claim 1, wherein analyzing image data of the image includes detecting the one or more objects in the captured image, and wherein the one or more information items corresponds to a recognition signature that identifies each of the detected one or more objects from other objects in other images of the collection.

4. The method of claim 1, wherein the one or more objects include a person.

5. The method of claim 1, wherein the one or more objects include an object selected from a group consisting of:

(i) a face of a person, (ii) clothing or apparel of the person, (iii) hair of the person, and (iv) a text in the image.

6. The method of claim 1, wherein the input corresponds to at least a portion of an image, and wherein correlating the input to at least a first information item includes determining a recognition signature for an object detected as being part of the input.

7. The method of claim 6, wherein the object corresponds to at least a portion of a person, and wherein the recognition signature identifies the person from other persons in other images in the collection.

8. The method of claim 7, wherein the object corresponds to at least one of (i) a face of the person, (ii) a hair of the person, and (iii) clothing or apparel of the person.

9. The method of claim 6, wherein the object corresponds to an item carrying text in the image, and wherein the recognition signature recognizes the text of the item in the image.

10. A method for enabling searching of a collection of captured images that form at least a portion of a library of images, the method comprising:

receiving image input from a user;

identifying a set of one or more captured images in the collection based on the image input.

11. The method of claim 10, wherein receiving image input includes identifying a portion of an image selected by the user.

12. The method of claim 10, wherein receiving image input includes presenting an image to a user having one or more selectable portions, and wherein receiving the image input includes receiving the user'"'"'s selection of one or more of the selectable portions.

13. The method of claim 10, further comprising detecting an object from the image input, wherein the object corresponds to at least a portion of a person.

14. The method of claim 13, wherein the object corresponds to at least one of (i) a face of the person, (ii) a hair of the person, and (iii) clothing or apparel of the person.

15. The method of claim 13, further comprising determining a recognition signature for the object, and wherein identifying a set of one or more captured images includes using the recognition signature.

16. The method of claim 10, further comprising detecting an object from the image input, wherein the object corresponds to an item carrying a text.

17. The method of claim 16, further comprising determining a recognition signature corresponding to a recognition of the text on the item.

18. A system for performing search of recognized images, the system comprising:

an image analysis module configured to analyze image data from at a set of captured images in order to generate recognition information for persons that appear in images in the set of captured images;

one or more indexes that store at least one of recognition information or correlation information based on the recognition information; and

a search module that is configured to perform a search operation based on a user-specified criteria using the one or more indexes.

19. The system of claim 18, wherein the recognition information includes quantitative data that is stored in the index, wherein the quantitative data represents recognition signatures of persons recognized by the image analysis module.

20. The system of claim 18, further comprising a search module that is configured to perform a search operation based on a user-specified criteria using the index.

21. The system of claim 20, further comprising a search module that is configured to perform a search operation based on a user-specified criteria, and wherein the user-specified criteria originates from an image, and is compared against quantitative information stored in the index.

22. The system of claim 21, wherein the user-specified criteria is based on an image specified by the user, and wherein the image analysis module generates a recognition signature for the specified image of the user, and wherein the search module identifies a result by comparing the recognition signature to other recognition signatures stored in the index.

23. The system of claim 22, wherein the search module identifies the result by identifying recognition signatures that are similar but not matching to the recognition signature for the image specified by the user.

24. The system of claim 22, wherein the search module identifies the result by identifying recognition signatures that are matching to the recognition signature for the image specified by the user.

25. The system of claim 18, wherein at least one of the image analysis module, the one or more index, and the search module are located on a server that communicates with a terminal of the user.

26. A computer-readable medium carrying instructions for perform steps comprising enabling searching of a collection of captured images that form at least a portion of a library of images, wherein the instructions include instructions that when executed by one or more processes, cause the one or more processors to perform steps comprising:

receiving image input from a user;

identifying a set of one or more captured images in the collection based on the image input.