Message suggestions

• US 9,860,200 B1
• Filed: 08/27/2014
• Issued: 01/02/2018
• Est. Priority Date: 08/27/2014
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Braun, Max Benjamin, Patel, Nirmal Jitendra
• Primary Examiner(s): Hill, Stanley K
• Assistant Examiner(s): Lee, Tsu-Chang
• Application Number:
  US14/470,904
• Time in Patent Office: 1,224 Days
• Field of Search: 706 12
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: H04L 51/02 using automatic reactions or user delegation, e.g. automatic replies or chatbot-generated messages
• CPC Class Codes: H04L 51/046 Interoperability with other network applications or services

Abstract

A method implemented at least in part on a head-mountable device (HMD) may involve: for each of one or more messages that are selected from a plurality of messages from an account: (a) extracting one or more phrases from a respective selected message; (b) determining that a conversation includes the selected message and one or more other messages from the plurality of messages; (c) generating a first feature vector based on the conversation; and (d) generating one or more training-data sets that include one or more of the phrases and the first feature vector. The method also includes training a machine-learning application with at least a portion of the one or more training-data sets, and applying the trained machine-learning application to process an incoming message and to determine one or more reply messages corresponding to the incoming message. The method may display the reply message(s) on a graphic display of the HMD.

Claims

1. A method comprising:

for each of one or more messages that are selected from a plurality of messages from an account;

(a) extracting one or more phrases from a respective selected message;

(b) determining that a conversation includes the respective selected message and one or more other messages from the plurality of messages;

(c) generating a first feature vector based on the conversation, wherein the first feature vector includes one or more first features, wherein the one or more first features include one or more words from the conversation; and

(d) generating, by a computing system, one or more training-data sets, wherein each training-data set comprises one of the phrases and the first feature vector;

training, by the computing system, a machine-learning application with at least a portion of the one or more training-data sets that are generated for the one or more selected messages;

applying the trained machine-learning application to process an incoming message to the account;

responsive to applying the trained machine-learning application, determining one or more reply messages corresponding to the incoming message, wherein the one or more reply messages include at least one of the extracted one or more phrases; and

displaying one or more cards on a display of a head-mountable device (HMD), wherein the one or more cards include at least a portion of the incoming message and the one or more reply messages corresponding to the incoming message.

2. The method of claim 1, further comprising receiving data input by the HMD indicating a selection of a particular one of the one or more reply messages and, responsively, sending the reply message and displaying a message card describing the reply message.

3. The method of claim 2, wherein receiving input via the HMD includes receiving input through one or more of a microphone, a touchpad disposed on the HMD, or an eye-tracking sensor.

4. The method of claim 1, wherein the one or more cards further includes a voice response command along with the one or more reply messages.

5. The method of claim 1, further comprising determining, by the computing system based on one or more of a number of the one or more reply messages or a length of the one or more reply messages, to arrange the one or more reply messages of the one or more cards in a top-level menu format or a submenu format.

6. The method of claim 1, wherein applying the trained machine-learning application to an incoming message further comprises:

processing, by the computing system, the incoming message to generate a second feature vector based on the incoming message, wherein the second feature vector includes one or more second features of the incoming message, wherein the one or more second features include one or more words;

processing, by the computing system using the trained machine-learning application, the second feature vector to identify one or more of the extracted phrases as the one or more reply messages; and

initiating, by the computing system, the display of the one or more reply messages on the display for selection to reply to the incoming message.

7. The method of claim 6, wherein generating the first feature vector includes identifying a sender of the one or more other messages in the conversation, wherein the one or more first features include an identity of the sender of the one or more other messages, wherein processing the incoming message includes identifying a sender of the incoming message, and wherein the one or more second features include an identity of the sender of the incoming message.

8. The method of claim 7, further comprising identifying one or more voice calls from the sender of the one or more other messages in the conversation, wherein the first feature vector includes information regarding the one or more voice calls.

9. The method of claim 1, wherein extracting one or more phrases includes normalizing the respective selected message by converting text to lowercase letters and removing punctuation, and using a natural language toolkit sentence tokenizer to extract the one or more phrases from the respective selected message.

10. The method of claim 1, wherein generating the first feature vector includes normalizing the one or more other messages in the conversation by converting words to lowercase letters, removing punctuation, removing stop words, and removing hash (#) characters.

11. The method of claim 10, wherein normalizing the one or more other messages includes converting ASCII emoticons into respective Unicode forms, and wherein the one or more first features include the Unicode forms of the ASCII emoticons.

12. The method of claim 10, wherein normalizing the one or more other messages includes categorizing images and Internet links as a generic feature, and wherein the one or more first features include the generic feature.

13. The method of claim 1, wherein generating the first feature vector includes identifying one or more times that correspond to when one or more other messages in the conversation were sent or received, and wherein the one or more first features include the identified one or more times.

14. The method of claim 1, further comprising removing, before training, one or more phrases from the training-data sets that appear fewer than a threshold number of times.

15. The method of claim 1, further comprising using metrics to determine the one or more reply messages corresponding to the incoming message, and wherein the metrics relate to one or more of how often a reply message is selected to respond to an incoming message or how often the reply responses are provided for selection.

16. The method of claim 1, further comprising:

processing, by the computing system, a message to be sent from the account to create a third feature vector that includes an identity of a recipient of the message to be sent as a feature;

processing, by the computing system using the trained machine-learning application, the third feature vector to determine one or more second messages that may be included in the message to be sent; and

initiating a display of the one or more second messages on the display.

17. A system comprising:

a non-transitory computer-readable medium; and

program instructions stored on the non-transitory computer-readable medium and executable by at least one processor to;

for each of one or more messages that are selected from a plurality of messages from an account;

(a) extract one or more phrases from a respective selected message;

(b) determine that a conversation includes the selected message and one or more other messages from the plurality of messages;

(c) generate a first feature vector based on the conversation, wherein the first feature vector includes one or more first features, wherein the one or more first features include one or more words from the conversation; and

(d) generate one or more training-data sets, wherein each training-data set comprises one of the phrases and the first feature vector;

train a machine-learning application with at least a portion of the one or more training-data sets that are generated for the one or more selected messages;

apply the trained machine-learning application to process an incoming message to the account;

responsive to applying the trained machine-learning application, determine one or more reply messages corresponding to the incoming message, wherein the one or more reply messages include at least one of the extracted one or more phrases; and

initiate a display of the one or more reply messages on a graphic display of a head-mountable device (HMD).

18. The system of claim 17, wherein the program instructions stored on the non-transitory computer-readable medium and executable by at least one processor to apply the trained machine-learning application to process an incoming message further include instructions to:

process the incoming message to create a second feature vector based on the incoming message, wherein the second feature vector includes one or more second features of the incoming message, wherein the one or more second features include one or more words;

process, using the trained machine-learning application, the second feature vector to identify one or more of the extracted phrases as the one or more reply messages; and

provide the one or more reply messages for selection to reply to the incoming message.

19. The system of claim 17, further comprising program instructions stored on the non-transitory computer-readable medium and executable by at least one processor to:

process a message to be sent from the account to create a third feature vector that includes an identity of a recipient of the message to be sent as a feature;

process, using the trained machine-learning application, the third feature vector to determine one or more second messages that may be included in the message to be sent; and

initiate a display of the one or more second messages on a graphic display.

20. A non-transitory computer-readable medium having stored thereon instructions executable by a computing device to cause the computing device to perform functions comprising:

for each of one or more messages that are selected from a plurality of messages from an account;

(a) extracting one or more phrases from a respective selected message;

(b) determining that a conversation includes the selected message and one or more other messages from the plurality of messages;

(c) generating a first feature vector based on the conversation, wherein the first feature vector includes one or more first features, wherein the one or more first features include one or more words from the conversation; and

(d) generating one or more training-data sets, wherein each training-data set comprises one of the phrases and the first feature vector;

training a machine-learning application with at least a portion of the one or more training-data sets that are generated for the one or more selected messages;

applying the trained machine-learning application to process an incoming message to the account;

responsive to applying the trained machine-learning application, determining one or more reply messages corresponding to the incoming message, wherein the one or more reply messages include at least one of the extracted one or more phrases; and

initiating a display of the one or more reply messages on a graphic display of a head-mountable device (HMD).

21. The non-transitory computer-readable medium of claim 20, wherein the instructions for applying the trained machine-learning application to an incoming message further comprises instructions executable by the computing device to cause the computing device to perform functions comprising:

processing the incoming message to create a second feature vector based on the incoming message, wherein the second feature vector includes one or more second features of the incoming message, wherein the one or more second features include one or more words;

processing, using the trained machine-learning application, the second feature vector to identify one or more of the extracted phrases as the one or more reply messages; and

providing the one or more reply messages for selection to reply to the incoming message.

22. The non-transitory computer-readable medium of claim 20, further comprising instructions executable by the computing device to cause the computing device to perform functions including:

processing a message to be sent from the account to create a third feature vector that includes an identity of a recipient of the message to be sent as a feature;

processing, using the trained machine-learning application, the third feature vector to determine one or more second messages that may be included in the message to be sent; and

initiating a display of the one or more second messages on the graphic display.