Automatic label suggestions for albums based on machine learning

• US 8,886,576 B1
• Filed: 08/14/2012
• Issued: 11/11/2014
• Est. Priority Date: 06/22/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Aradhye, Hrishikesh, Carceroni, Rodrigo, Sanketi, Pannag Raghunath
• Primary Examiner(s): Chaki, Kakali
• Assistant Examiner(s): PELLETT, DANIEL T
• Application Number:
  US13/585,458
• Time in Patent Office: 819 Days
• Field of Search: None
• US Class Current: 706/12
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: H04M 1/6008 in the transmitter circuit
• CPC Class Codes: H04M 1/72454 according to context-related or environment-related conditions

Abstract

Methods and apparatus for suggesting image, video, and image album titles are presented. A machine-learning service executing on a mobile platform receives feature-related data. The feature-related data includes image-related data related to one or more images received from an application executing on the mobile platform and platform-related data received from the mobile platform. The image-related data and the platform-related data differ. The machine-learning service generates a title related to the one or more images by performing a machine-learning operation on the feature-related data. The machine-learning service sends the title related to the one or more images to the application.

Claims

1. A method, comprising:

receiving feature-related data for a machine-learning service executing on a mobile platform, wherein the feature-related data comprises image-related data related to at least a first image received from an application executing on the mobile platform and platform-related data received from the mobile platform, and wherein the image-related data and the platform-related data differ;

generating a first title and a second title related to the first image by the machine-learning service performing a machine-learning operation on the feature-related data, wherein at least one of the first title and the second title comprises a name of a person;

sending the first title and the second title related to the one or more images from the machine-learning service to the application based on a rate of notification;

receiving an indication of selection of the first title at the machine-learning service using a graphical user interface associated with the application; and

after receiving the indication of selection, the machine learning service;

determining an amount of elapsed time for user response associated with the indication of selection;

determining whether the amount of elapsed time exceeds a threshold amount of time;

after determining that the amount of elapsed time exceeds the threshold amount of time, decreasing the rate of notification;

increasing a probability of generating a title similar to the first title, and decreasing a probability of generating a title similar to the second title;

receiving second image-related data related to at least a second image, wherein the second image differs from the first image;

generating at least a third title for the second image based on the probability of generating a title similar to the first title and the probability generating a title similar to the second title; and

sending the at least the third title for the second image from the machine-learning service to the application based on the rate of notification.

2. The method of claim 1, wherein the image-related data is generated by an image-identification system (IIS) executing on the mobile platform.

3. The method of claim 2, wherein the IIS is configured to extract image-related features from the first image.

4. The method of claim 3, wherein the IIS is further configured to classify at least one image-related feature of the image-related features.

5. The method of claim 4, wherein the IIS is further configured to recognize the at least one classified image-related feature.

6. The method of claim 5, wherein the at least one classified image-related feature comprises a feature related to a face of the person, and wherein the IIS is configured to recognize the face of the person.

7. The method of claim 5, wherein the at least one classified image-related feature comprises a feature related to a landmark, and wherein the IIS is configured to recognize the landmark.

8. The method of claim 5, wherein the at least one classified image-related feature includes a feature related to an object, and wherein the image-processing function is configured to recognize the object.

9. The method of claim 1, wherein the title further comprises at least one name from the image-related data and at least one datum from the platform-related data.

10. The method of claim 9, wherein the at least one name from the image-related data comprises at least one name selected from a name of a landmark and a name of an object, and

wherein the at least one datum from the platform-related data comprises at least one datum selected from among a date, a time, a calendar entry, a social-networking message, and a location.

11. An article of manufacture including a non-transitory computer-readable storage medium having instructions stored thereon that, when executed by a processor, cause the processor to perform functions comprising:

receiving feature-related data for a machine-learning service, wherein the feature-related data comprises image-related data related to at least a first image and platform-related data, and wherein the image-related data and the platform-related data differ;

generating a first title and a second title related to the first image by the machine-learning service by performing a machine-learning operation on the feature-related data, wherein at least one of the first title and the second title comprises a name of a person;

sending the first title and the second title related to the one or more images for the image to the application based on a rate of notification;

receiving an indication of selection of the first title using a graphical user interface; and

after receiving the indication of selection;

determining an amount of elapsed time for user response associated with the indication of selection;

determining whether the amount of elapsed time exceeds a threshold amount of time;

after determining that the amount of elapsed time exceeds the threshold amount of time, decreasing the rate of notification;

increasing a probability of generating a title similar to the first title, and decreasing a probability of generating a title similar to the second title;

receiving second image-related data related to at least a second image, wherein the second image differs from the first image;

generating at least a third title for the second image based on the probability of generating a title similar to the first title and the probability generating a title similar to the second title; and

sending the at least the third title for the second image from the machine-learning service to the application based on the rate of notification.

12. The article of manufacture of claim 11, wherein the image-related data is generated by an image-processing function configured to:

extract image-related features from the first image,

classify at least one image-related feature of the image-related features, and

recognize the at least one classified image-related feature.

13. The article of manufacture of claim 12, wherein the at least one classified image-related feature includes a feature related to a face of the person, and wherein the image-processing function is configured to recognize the face of the person.

14. The article of manufacture of claim 12, wherein the at least one classified image-related feature includes a feature related to a landmark, and wherein the image-processing function is configured to recognize the landmark.

15. The article of manufacture of claim 12, wherein the at least one classified image-related feature includes a feature related to an object, and wherein the image-processing function is configured to recognize the object.

16. The article of manufacture of claim 11, wherein the title further comprises at least one name from the image-related data and at least one datum from the platform related data,

wherein the at least one name from the image-related data comprises at least one name selected from a name of a landmark and a name of an object, and

wherein the at least one datum from the platform-related data comprises at least one datum selected from among a date, a time, a calendar entry, a social-networking message, and a location.

17. A mobile platform, comprising:

a processor; and

a non-transitory computer-readable storage medium, configured to store instructions that, when executed by the processor, cause the mobile platform to perform functions comprising;

receiving feature-related data for a machine-learning service, wherein the feature-related data comprises image-related data related to at least a first image and platform-related data, and wherein the image-related data and the platform-related data differ;

generating a first title and a second title related to the first image by the machine-learning service by performing a machine-learning operation on the feature-related data, wherein at least one of the first title and the second title comprises a name of a person;

sending the first title and the second title related to the one or more images for the image to the application based on a rate of notification;

receiving an indication of selection of the first title using a graphical user interface; and

after receiving the indication of selection;

determining an amount of elapsed time for user response associated with the indication of selection;

determining whether the amount of elapsed time exceeds a threshold amount of time;

after determining that the amount of elapsed time exceeds the threshold amount of time, decreasing the rate of notification;

increasing a probability of generating a title similar to the first title, and decreasing a probability of generating a title similar to the second title;

receiving second image-related data related to at least a second image, wherein the second image differs from the first image;

generating at least a third title for the second image based on the probability of generating a title similar to the first title and the probability generating a title similar to the second title; and

sending the at least the third title for the second image from the machine-learning service to the application based on the rate of notification.

18. The mobile platform of claim 17, wherein the image-related data is generated by an image-processing function configured to:

extract image-related features from the first image,

classify at least one image-related feature of the image-related features, and

recognize the at least one classified image-related feature.

19. The mobile platform of claim 18, wherein the at least one classified image-related feature includes a feature related to a face of the person, and wherein the image-processing function is configured to recognize the face of the person.

20. The mobile platform of claim 17,

wherein the title further comprises at least one name from the image-related data and at least one datum from the platform related data,

wherein the at least one name from the image-related data comprises at least one name selected from a name of a landmark and a name of an object, and

wherein the at least one datum from the platform-related data comprises at least one datum selected from among a date, a time, a calendar entry, a social-networking message, and a location.