Providing suggestions within a document

• US 10,223,392 B1
• Filed: 02/22/2018
• Issued: 03/05/2019
• Est. Priority Date: 05/12/2014
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Gubin, Maxim, Dauber, Kenneth W., Bharat, Krishna, Sung, Sang Soo
• Primary Examiner(s): Hoang, Son T
• Application Number:
  US15/902,103
• Time in Patent Office: 376 Days
• Field of Search: None
• CPC Class Codes: G06F 16/219 Managing data history or versioning querying versioned data G06F16/2474; querying temporal data G06F16/2477
• CPC Class Codes: G06F 16/2465 Query processing support for facilitating data mining operations in structured databases
• CPC Class Codes: G06F 16/93 Document management systems

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for providing suggestions within a document. In one aspect, a method includes obtaining textual input provided to a document editing application by a user device, the textual input being provided to the document editing application for inclusion in a document; identifying performance measures associated with the current editing session for the document, each performance measure being based on session data obtained from the user device during a document editing session, the session data being for the textual input and prior text that was included in the document prior to the textual input; providing the performance measures as input to a suggestion model that was trained using historical performance measures identified in performance logs for historical document editing sessions of users; and throttling textual suggestions during the current editing session based on the output of the suggestion model.

Claims

1. A method implemented by a data processing apparatus, the method comprising:

obtaining, during a current editing session of a document, textual input provided to a document editing application for inclusion in the document by a user associated with a user device;

identifying, during the current editing session, one or more performance measures associated with the current editing session of the document, each performance measure being based on session data obtained from the user device and at least one of the performance measures being typing speed of the user during the current editing session, the session data including data related to (i) the obtained textual input and (ii) prior textual data that was included in the document prior to the obtained textual input;

receiving textual suggestions based on the textual input, each textual suggestion received from a suggestion model and having a respective confidence score, each textual suggestion comprising one or more words to be suggested for inclusion in the document;

providing the one or more performance measures as input to the suggestion model that has been trained using historical performance measures identified in performance logs for a plurality of historical document editing sessions of a plurality of users and prior to the current editing session;

generating, by the suggestion model, a threshold value based on the one or more performance measures that were provided as an input to the suggestion model; and

providing, during the current editing session, textual suggestions that have respective confidence scores greater than the threshold value.

2. The method of claim 1, the method further comprising:

training the suggestion model, wherein training the suggestion model comprises training the suggestion model based on historical suggestions included in the performance logs.

3. The method of claim 2, wherein the method further comprises:

determining, based on the historical performance measures and the historical suggestions included in the performance logs, a relation between the user'"'"'s typing speed and a rate of historical suggestion acceptance; and

wherein training the suggestion model further comprises;

training the suggestion model based on the determined relation.

4. The method of claim 2, wherein the method further comprises:

determining, based on the historical performance measures and the historical suggestions included in the performance logs, a relation between a suggestion confidence score and a rate of historical suggestion acceptance; and

wherein training the suggestion model further comprises;

training the suggestion model based on the determined relation.

5. The method of claim 1, wherein the document editing application is an e-mail application.

6. The method of claim 1, wherein the one or more performance measures further comprise a rate of suggestion acceptance that specifies a rate of acceptance for previous suggestions that were provided during the plurality of historical document editing sessions.

7. The method of claim 1, wherein the document editing application is one of a plurality of document editing applications, and the suggestion model was trained using respective historical performance measures identified in respective performance logs for the document editing application.

8. A system comprising:

one or more data processing apparatus implemented at least partially by hardware; and

a data storage device storing instructions that, when executed by the one or more data processing apparatus, cause the one or more data processing apparatus to perform operations comprising;

obtaining, during a current editing session of a document, textual input provided to a document editing application for inclusion in the document by a user associated with a user device;

identifying, during the current editing session, one or more performance measures associated with the current editing session of the document, each performance measure being based on session data obtained from the user device and at least one of the performance measures being typing speed of the user during the current editing session, the session data including data related to (i) the obtained textual input and (ii) prior textual data that was included in the document prior to the obtained textual input;

receiving textual suggestions based on the textual input, each textual suggestion received from a suggestion model and having a respective confidence score, each textual suggestion comprising one or more words to be suggested for inclusion in the document;

providing the one or more performance measures as input to the suggestion model that has been trained using historical performance measures identified in performance logs for a plurality of historical document editing sessions of a plurality of users and prior to the current editing session;

generating, by the suggestion model, a threshold value based on the one or more performance measures that were provided as an input to the suggestion model; and

providing, during the current editing session, textual suggestions that have respective confidence scores greater than the threshold value.

9. The system of claim 8, the method further comprising:

training the suggestion model, wherein training the suggestion model comprises training the suggestion model based on historical suggestions included in the performance logs.

10. The system of claim 9, wherein the method further comprises:

determining, based on the historical performance measures and the historical suggestions included in the performance logs, a relation between the user'"'"'s typing speed and a rate of historical suggestion acceptance; and

wherein training the suggestion model further comprises;

training the suggestion model based on the determined relation.

11. The system of claim 9, wherein the method further comprises:

determining, based on the historical performance measures and the historical suggestions included in the performance logs, a relation between a suggestion confidence score and a rate of historical suggestion acceptance; and

wherein training the suggestion model further comprises;

training the suggestion model based on the determined relation.

12. The system of claim 8, wherein the one or more performance measures further comprise a rate of suggestion acceptance that specifies a rate of acceptance for previous suggestions that were provided during the plurality of historical document editing sessions.

13. The system of claim 8, wherein the document editing application is one of a plurality of document editing applications, and the suggestion model was trained using respective historical performance measures identified in respective performance logs for the document editing application.

14. A non-transitory computer-readable medium comprising instructions that, when executed by one or more data processing apparatus, cause the one or more data processing apparatus to perform operations comprising:

obtaining, during a current editing session of a document, textual input provided to a document editing application during a current editing session for inclusion in the document by a user associated with a user device;

identifying, during the current editing session, one or more performance measures associated with the current editing session of the document, each performance measure being based on session data obtained from the user device and at least one of the performance measures being typing speed of the user during the current editing session, the session data including data related to (i) the obtained textual input and (ii) prior textual data that was included in the document prior to the obtained textual input;

receiving textual suggestions based on the textual input, each textual suggestion received from a suggestion model and having a respective confidence score, each textual suggestion comprising one or more words to be suggested for inclusion in the document;

providing the one or more performance measures as input to the suggestion model that has been trained using historical performance measures identified in performance logs for a plurality of historical document editing sessions of a plurality of users and prior to the current editing session;

generating, by the suggestion model, a threshold value based on the one or more performance measures that were provided as an input to the suggestion model; and

providing, during the current editing session, textual suggestions that have respective confidence scores greater than the threshold value.

15. The computer-readable medium of claim 14, the method further comprising:

training the suggestion model, wherein training the suggestion model comprises training the suggestion model based on historical suggestions included in the performance logs.

16. The computer-readable medium of claim 15, wherein the method further comprises:

determining, based on the historical performance measures and the historical suggestions included in the performance logs, a relation between the user'"'"'s typing speed and a rate of historical suggestion acceptance; and

wherein training the suggestion model further comprises;

training the suggestion model based on the determined relation.

17. The computer-readable medium of claim 15, wherein the method further comprises:

determining, based on the historical performance measures and the historical suggestions included in the performance logs, a relation between a suggestion confidence score and a rate of historical suggestion acceptance; and

wherein training the suggestion model further comprises;

training the suggestion model based on the determined relation.

18. The computer-readable medium of claim 14, wherein the one or more performance measures further comprise a rate of suggestion acceptance that specifies a rate of acceptance for previous suggestions that were provided during the plurality of historical document editing sessions.

19. The computer-readable medium of claim 14, wherein the document editing application is one of a plurality of document editing applications, and the suggestion model was trained using respective historical performance measures identified in respective performance logs for the document editing application.