Methods and apparatus for determining non-textual reply content for inclusion in a reply to an electronic communication

• US 10,454,861 B2
• Filed: 05/31/2018
• Issued: 10/22/2019
• Est. Priority Date: 01/01/2016
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Miklos, Balint, Emeagwali, Ijeoma, Sharp, Phillip, Raghavan, Prabhakar
• Primary Examiner(s): Nguyen, Thu Ha T
• Application Number:
  US15/994,789
• Publication Number:
  US 20180278560A1
• Time in Patent Office: 509 Days
• Field of Search: 709205, 709206, 709203
• CPC Class Codes: G06F 16/00 Information retrieval; Database structures therefor; File system structures therefor
• CPC Class Codes: G06F 16/903 Querying for retrieval from the web G06F16/953
• CPC Class Codes: G06F 40/279 Recognition of textual entities
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06Q 10/107 Computer-aided management of electronic mailing [e-mailing]
• CPC Class Codes: H04L 51/02 using automatic reactions or user delegation, e.g. automatic replies or chatbot-generated messages
• CPC Class Codes: H04L 51/08 Annexed information, e.g. attachments

Abstract

Methods and apparatus related to determining non-textual reply content for a reply to an electronic communication and providing the non-textual reply content for inclusion in the reply. Some of those implementations are directed to determining, based on an electronic communication sent to a user, one or more electronic documents that are responsive to the electronic communication, and providing one or more of those electronic documents for inclusion in a reply by the user to the electronic communication. For example, the electronic documents may be automatically attached to the reply and/or link(s) to the electronic documents automatically provided in the reply.

Claims

1. A method implemented by one or more processors, the method comprising:

determining a message feature of an electronic communication sent to a user;

issuing a search of one or more corpuses of electronic documents using a search parameter for the search that is based on the message feature of the electronic communication;

receiving, in response to issuing the search, an indication of a subgroup of one or more of the electronic documents of the one or more corpuses that are responsive to the search;

selecting, based on receiving the indication, at least one selected electronic document of the electronic documents of the subgroup;

causing a selectable graphical element, corresponding to the at least one selected electronic document, to be presented at a client device of the user while the electronic communication is also being presented at the client device,

wherein causing the selectable graphical element to be presented at the client device comprises causing the selectable graphical element to be presented before any user interface input, that indicates a desire to reply to the electronic communication, has been provided at the client device;

receiving a selection of the selectable graphical element, the selection being by the user via a user interface input device of the client device; and

in response to receiving the selection;

automatically attaching the at least one selected electronic document to a reply electronic communication that is a reply to the electronic communication, or

automatically incorporating a link to the at least one selected electronic document into the reply electronic communication.

2. The method of claim 1, wherein the reply electronic communication is an editable reply electronic communication and further comprising:

in response to receiving the selection, causing the editable reply electronic communication to be automatically presented at the client device, wherein the presented editable reply electronic communication includes the automatically attached at least one selected electronic document or the automatically incorporated link.

3. The method of claim 1, further comprising:

providing, as input to a trained machine learning system, at least the message feature of the electronic communication; and

receiving, as output from the trained machine learning system that is responsive to the input, a likelihood that the electronic communication will be replied to with non-textual reply content;

wherein causing the selectable graphical element to be presented at the client device is based on the likelihood that the electronic communication will be replied to with non-textual reply content.

4. The method of claim 1, further comprising:

selecting, based on receiving the indication, at least one additional selected electronic document of the electronic documents of the subgroup; and

causing an additional selectable graphical element, corresponding to the at least one additional selected electronic document, to be presented at the client device while the electronic communication and the selectable graphical element are also being presented at the client device;

wherein selection of the additional selectable graphical element, in lieu of the selection of the selectable graphical element, causes automatic attachment of the at least one additional selected electronic document to the reply electronic communication or automatic incorporation of an alternative link, to the at least one additional selected electronic document, into the reply electronic communication.

5. The method of claim 1, wherein at least one of the one or more corpuses is not publicly accessible, but is accessible to the user.

6. The method of claim 5, wherein the at least one of the one or more corpuses is accessible only to the user and to one or more additional users or systems authorized by the user.

7. The method of claim 1, wherein issuing the search of the one or more corpuses comprises issuing a search of one or more indexes that index the electronic documents of the one or more corpuses.

8. The method of claim 1, wherein the subgroup includes a plurality of the electronic documents and further comprising:

receiving, in response to issuing the search, search rankings for the subgroup of the electronic documents of the one or more corpuses that are responsive to the search;

wherein selecting the at least one selected electronic document of the electronic documents of the subgroup is further based on the search rankings for the subgroup of the electronic documents.

9. The method of claim 1, wherein the message feature is an embedding vector of one or more features from the electronic communication.

10. A method implemented by one or more processors, the method comprising:

identifying an electronic communication sent to a user;

determining a likelihood that the electronic communication will be replied to with non-textual reply content, wherein determining the likelihood comprises;

providing, as input to a trained machine learning system, at least one message feature of the electronic communication; and

receiving, as output from the trained machine learning system that is responsive to the input, the likelihood that the electronic communication will be replied to with non-textual reply content;

in response to the likelihood satisfying a threshold;

causing a selectable graphical element, corresponding to at least one electronic document that is relevant to the electronic communication, to be presented at a client device of the user while the electronic communication is also being presented at the client device;

receiving a selection of the selectable graphical element, the selection being by the user via a user interface input device of the client device; and

in response to receiving the selection, performing one or both of;

automatically attaching the at least one selected electronic document to a reply electronic communication that is a reply to the electronic communication, and

automatically incorporating a link to the at least one selected electronic document into the reply electronic communication.

11. The method of claim 10, wherein the reply electronic communication is an editable reply electronic communication and further comprising:

in response to receiving the selection, causing the editable reply electronic communication to be automatically presented at the client device, wherein the presented editable reply electronic communication includes the automatically attached at least one selected electronic document or the automatically incorporated link.

12. The method of claim 10, further comprising:

in response to the likelihood satisfying the threshold;

causing an additional selectable graphical element, corresponding to the at least one additional selected electronic document, to be presented at the client device while the electronic communication and the selectable graphical element are also being presented at the client device;

wherein selection of the additional selectable graphical element, in lieu of the selection of the selectable graphical element, causes automatic attachment of the at least one additional selected electronic document to the reply electronic communication or automatic incorporation of an alternative link, to the at least one additional selected electronic document, into the reply electronic communication.

13. The method of claim 10, wherein the message feature is an embedding vector of one or more features from the electronic communication.

14. The method of claim 10, wherein causing the selectable graphical element to be presented at the client device comprises causing the selectable graphical element to be presented before any user interface input, that indicates a desire to reply to the electronic communication, has been provided at the client device.

15. The method of claim 10, wherein automatically incorporating the link to the at least one selected electronic document into the reply electronic communication is performed in response to receiving the selection.

16. A system, comprising:

an electronic communication stored in one or more non-transitory computer readable media, the electronic communication sent to a user;

at least one processor; and

memory coupled to the processor, wherein the memory stores instructions to be executed by the processor to perform steps comprising;

determining a likelihood that the electronic communication will be replied to with non-textual reply content, wherein determining the likelihood comprises;

providing, as input to a trained machine learning system, at least one message feature of the electronic communication; and

receiving, as output from the trained machine learning system that is responsive to the input, the likelihood that the electronic communication will be replied to with non-textual reply content;

in response to the likelihood satisfying a threshold;

causing a selectable graphical element, corresponding to at least one electronic document that is relevant to the electronic communication, to be presented at a client device of the user while the electronic communication is also being presented at the client device;

receiving a selection of the selectable graphical element, the selection being by the user via a user interface input device of the client device; and

in response to receiving the selection;

automatically attaching the at least one selected electronic document to a reply electronic communication that is a reply to the electronic communication, or

automatically incorporating a link to the at least one selected electronic document into the reply electronic communication.

17. The system of claim 16, wherein the reply electronic communication is an editable reply electronic communication and wherein the steps further comprise:

in response to receiving the selection, causing the editable reply electronic communication to be automatically presented at the client device, wherein the presented editable reply electronic communication includes the automatically attached at least one selected electronic document or the automatically incorporated link.

18. The system of claim 16, wherein causing the selectable graphical element to be presented at the client device comprises causing the selectable graphical element to be presented before any user interface input, that indicates a desire to reply to the electronic communication, has been provided at the client device.

19. The system of claim 16, wherein the steps further comprise:

in response to the likelihood satisfying the threshold;

causing an additional selectable graphical element, corresponding to the at least one additional selected electronic document, to be presented at the client device while the electronic communication and the selectable graphical element are also being presented at the client device;

wherein selection of the additional selectable graphical element, in lieu of the selection of the selectable graphical element, causes automatic attachment of the at least one additional selected electronic document to the reply electronic communication or automatic incorporation of an alternative link, to the at least one additional selected electronic document, into the reply electronic communication.