System and method for inputting text into electronic devices based on text and text category predictions

• US 9,046,932 B2
• Filed: 10/11/2010
• Issued: 06/02/2015
• Est. Priority Date: 10/09/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Touchtype Limited (Microsoft Corporation)
• Inventors: Medlock, Benjamin William, Reynolds, Jonathan Paul
• Primary Examiner(s): CHANG, LI WU
• Application Number:
  US13/501,087
• Publication Number:
  US 20120197825A1
• Time in Patent Office: 1,695 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/0237 using prediction or retrieval techniques
• CPC Class Codes: G06N 3/084 Backpropagation, e.g. using gradient descent

Abstract

The present invention relates generally to a system and method for text prediction and, in particular, a system and method for the adaptive weighting of text predictions. The present invention provides a modular language model-based text prediction system for the adaptive weighting of text prediction components and a method for doing so. The system uses category predictions, generated by a timed aggregated perceptron classifier, to weight, using a weighting module, text predictions generated by a text prediction engine comprising at least one predictor.

Claims

1. A system for generating text input, the system comprising:

a processor; and

memory storing instructions that, when executed by the processor, configure the processor to;

receive a text input;

generate, using at least two predictors, a plurality of text predictions from each predictor based on the text input, wherein each of the plurality of text predictions is a word or phrase mapped to a probability value;

generate, using a classifier, at least two text category predictions based on the text input, wherein each text category prediction comprises a category mapped to a probability value;

weight, using a weighting module, the probabilities of the plurality of text predictions from each predictor by the probability of a corresponding category prediction to generate a plurality of category-weighted text predictions;

determine the n most probable category-weighted text predictions; and

output the n most probable category-weighted text predictions for presentation to a user.

2. The system of according to claim 1, wherein the at least two predictors comprise a plurality of predictors, each of the plurality of predictors trained by a separate text source.

3. The system according to claim 2, further comprising instructions that, when executed by the processor, configure the processor to generate a feature vector representing the text input by extracting features from the input text, calculating the term frequency-inverse document frequency for each feature in the input text, and normalising the resulting vector to unit length.

4. The system according to claim 1, wherein the instructions configure the processor to generate a feature vector for a category by extracting a set of features from a text source associated with the category, calculating the term frequency-inverse document frequency for each feature in the text source, and normalising the resulting vectors to unit length.

5. The system according to claim 4, further comprising instructions that, when executed by the processor, configure the processor to train the classifier on a plurality of feature vectors for a plurality of categories.

6. The system according to claim 1, wherein the classifier is configured to generate a confidence vector relating to each of the at least two categories.

7. The system according to claim 6, wherein the weighting module is configured to generate a weights vector from the confidence vector by setting a largest positive value in the confidence vector to 1, and dividing all other positive values in the confidence vector by the largest positive value in the confidence vector multiplied by a constant factor, and by setting any negative confidence value to zero.

8. The system according to claim 7, wherein the weighting module is configured to scale the text predictions generated by the text prediction engine by the weights vector to generate the subset of text predictions.

9. The system according to claim 1, at least one of the at least two predictors comprises an adaptive prediction system.

10. The system according to claim 9, wherein the at least one adaptive prediction system comprises:

at least one additional predictor;

a second classifier;

a second weighting module; and

instructions that, when executed by the processor, configure the processor to;

receive the input text;

generate text predictions using the at least one additional predictor and the text input;

generate, using the second classifier, at least one text category prediction based on the text input; and

weight, using the second weighting module, the text predictions from the at least one additional predictor by the at least one category prediction from the second classifier to generate new text predictions.

11. A method of generating text predictions, comprising;

receiving a text input;

generating using at least two predictors a plurality of the text predictions from each predictor based upon the text input, wherein each of the plurality of text predictions is a word or phrase mapped to a probability value;

generating, using a classifier, at least two text category predictions based on the text input and a feature vector for a category, wherein each text category prediction comprises a category mapped to a probability value;

weighting the probabilities of the plurality of text predictions from each predictor by the probability of a corresponding text category prediction to generate a plurality of category-weighted text predictions;

determining the n most probable category-weighted text predictions; and

outputting the n most probable category-weighted text predictions.

12. The method according to claim 11, wherein the text predictions are generated by a plurality of predictors and the plurality of predictors is trained, each predictor being trained based upon a separate text source.

13. The method according to claim 12, further comprising generating a feature vector representing the text input into a device by a user by extracting features from the text input, calculating the term frequency-inverse document frequency for each feature in the text input, and normalising the resulting vector to unit length.

14. The method according to claim 11, further comprising generating a feature vector for a category by extracting a set of features from a text source associated with the category, calculating the term frequency-inverse document frequency for each feature in the text source, and normalising the resulting vectors to unit length.

15. The method according to claim 11, further comprising generating a set of text category predictions based upon the text input and a plurality of feature vectors for a plurality of categories.

16. The method according to claim 15, wherein the method further comprises training, using a processor, the classifier based upon the plurality of feature vectors.

17. The method according to claim 11, wherein the step of generating the at least one text category prediction comprises generating a confidence vector relating to each of the at least two categories.

18. The method according to claim 17, wherein the step of weighting the plurality of text predictions based on the at least one text category prediction comprises generating a weights vector from the confidence vector by setting a largest positive value in the confidence vector to 1, and dividing all other positive values in the confidence vector by the largest positive value in the confidence vector multiplied by a constant factor, and by setting any negative confidence value to zero.

19. The method according to claim 18, wherein the step of weighting the plurality of text predictions based on the at least one text category prediction comprises scaling the text predictions generated by the text prediction engine by the weights vector.