ORGANIZING IMAGES ASSOCIATED WITH A USER

• US 20170185869A1
• Filed: 12/28/2015
• Published: 06/29/2017
• Est. Priority Date: 12/28/2015
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: RAVI, Sujith, DUA, Robin
• Granted Patent: US 9,881,236 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06Q 30/0633 Lists, e.g. purchase orders, compilation or processing

Abstract

A method includes identifying images associated with a user, where the image is identified as at least one of captured by a user device associated with the user, stored on the user device associated with the user, and stored in cloud storage associated with the user. The method also includes for each of the images, determining one or more labels, wherein the one or more labels are based on at least one of metadata and a primary annotation. The method also includes generating a mapping of the one or more labels to one or more confidence scores, wherein the one or more confidence scores indicate an extent to which the one or more labels apply to corresponding images. The method also includes interacting with the user to obtain identifying information that is used to categorize one or more of the images.

Claims

1. A computer-implemented method comprising:

identifying an image associated with a user, wherein the image is identified as at least one of captured by a user device associated with the user, stored on the user device associated with the user, and stored in cloud storage associated with the user;

for each of the images, determining one or more labels, wherein the one or more labels are based on at least one of metadata and a primary annotation;

generating a mapping of the one or more labels to one or more confidence scores, wherein the one or more confidence scores indicate an extent to which the one or more labels apply to corresponding images; and

interacting with the user to obtain identifying information that is used to categorize one or more of the images.

2. The method of claim 1, further comprising:

updating the mapping by adding the identifying information to the one or more of the images.

3. The method of claim 1, further comprising:

receiving one or more user-generated search terms from the user for one or more of the images associated with the user;

determining an ambiguity in the user-generated search terms; and

requesting the user to provide clarifying information to resolve the ambiguity in the user-generated search terms.

4. The method of claim 1, further comprising:

receiving an instruction from the user to send one or more of the images to a person associated with a third-party application, determine a contact profile associated with the person; and

providing the person with the one or more images based on the contact profile.

5. The method of claim 1, wherein the one or more labels includes a secondary annotation and the secondary annotation is generated by performing label expansion on at least one of the metadata and the primary annotation.

6. The method of claim 1, wherein interacting with the user includes transmitting an image and an audio prompt to provide the identifying information about an entity in the image.

7The. The method of claim 1, further comprising:

receiving a request from the user to purchase an item related to an entity in one of the images; and

purchasing the item for the user.

8. A computer system comprising:

one or more processors coupled to a memory;

an image processing module stored in the memory and executable by the one or more processors, the image processing module operable to identify images associated with a user and, for each of the images, determine one or more labels, wherein the one or more labels are based on at least one of metadata, a primary annotation, and a secondary annotation and the secondary annotation is generated by performing label expansion on at least one of the metadata and the primary annotation;

an indexing module stored in the memory and executable by the one or more processors, the indexing module operable to generate a mapping of the one or more labels to one or more confidence scores, wherein the one or more confidence scores indicate an extent to which the one or more labels apply to corresponding images; and

an image assistant stored in the memory and executable by the one or more processors, the image assistant operable to receive a first search query that includes one or more user-generated search terms that correspond to one or more of the one or more labels and provide the user with a corresponding image.

9. The system of claim 8, wherein the indexing module is further operable to:

interact with the user to obtain identifying information that is used to categorize one or more of the images; and

instruct the indexing module to update the mapping by adding the identifying information to the one or more of the images.

10. The system of claim 8, further comprising:

a search module stored in the memory and executable by the one or more processors, the search module operable to receive a second request from the user that includes one or more user-generated search terms for one or more of the images associated with the user; and

wherein the image assistant is further operable to determine an ambiguity in the user-generated search terms and to request the user to provide clarifying information to resolve the ambiguity in the user-generated search terms.

11. The system of claim 8, wherein the image assistant is further operable to receive an instruction from the user to send one or more of the images to a person associated with a third-party application, determine a contact profile associated with the person, and provide the person with the one or more images based on the contact profile.

12. The system of claim 8, wherein the one or more labels includes a secondary annotation and the secondary annotation is generated by performing label expansion on at least one of the metadata and the primary annotation.

13. The system of claim 8, wherein interacting with the user includes transmitting an image and an audio prompt to provide the identifying information about an entity in the image.

14. The system of claim 8, wherein the image assistant is further operable to:

receive a request from the user to purchase an item related to an entity in one of the images; and

purchase the item for the user.

15. A non-transitory computer storage medium encoded with a computer program, the computer program comprising instructions that, when executed by one or more computers, cause the one or more computers to perform operations comprising:

identifying an image associated with a user, wherein the image is identified as at least one of captured by a user device associated with the user, stored on the user device associated with the user, and stored in cloud storage associated with the user;

for each of the images, determining one or more labels, wherein the one or more labels are based on at least one of metadata and a primary annotation;

generating a mapping of the one or more labels to one or more confidence scores, wherein the one or more confidence scores indicate an extent to which the one or more labels apply to corresponding images; and

interacting with the user to obtain identifying information that is used to categorize one or more of the images.

16. The computer storage medium of claim 15, wherein the instructions are further operable to perform operations comprising:

updating the mapping by adding the identifying information to the one or more of the images.

17. The computer storage medium of claim 15, wherein the instructions are further operable to perform operations comprising:

receiving one or more user-generated search terms from the user for one or more of the images associated with the user;

determining an ambiguity in the user-generated search terms; and

requesting the user to provide clarifying information to resolve the ambiguity in the user-generated search terms.

18. The computer storage medium of claim 15, wherein the instructions are further operable to perform operations comprising:

receive an instruction from the user to send one or more of the images to a person associated with a third-party application, determine a contact profile associated with the person; and

provide the person with the one or more images based on the contact profile.

19. The computer storage medium of claim 15, wherein the one or more labels includes a secondary annotation and the secondary annotation is generated by performing label expansion on at least one of the metadata and the primary annotation.

20. The computer storage medium of claim 15, wherein interacting with the user includes transmitting an image and an audio prompt to provide the identifying information about an entity in the image.