System and method for inputting text into small screen devices

• US 9,189,472 B2
• Filed: 03/08/2012
• Issued: 11/17/2015
• Est. Priority Date: 03/30/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Touchtype Limited (Microsoft Corporation)
• Inventors: Medlock, Benjamin William, Reynolds, Jonathan Paul
• Primary Examiner(s): He, Jialong
• Application Number:
  US13/415,574
• Publication Number:
  US 20120223889A1
• Time in Patent Office: 1,349 Days
• Field of Search: 715/773
• US Class Current: 1/1
• CPC Class Codes: G06F 3/0237 using prediction or retrieval techniques
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06F 40/274 Converting codes to words; Guess-ahead of partial word inputs

Abstract

An embodiment is directed to an interface for a small screen device, such as a watch, that enables a user to enter text on the small screen device by touching in the vicinity of characters, rather than aiming for a particular button or the exact location of a character. Embodiments further enable the design of interfaces without the use of buttons for controlling the entry of text on the small screen device.

Claims

1. A non-transitory computer readable medium embodying a program of instructions executable by a device to perform program steps for operating a touch-screen interface on the device, the program steps comprising:

displaying a character pane on the touch-screen interface, the character pane defining a plurality of targets corresponding to a plurality of characters, numerals, punctuation marks, or combinations thereof;

accepting a current touch-input event, the current touch-input event corresponding to a coordinate location on the touch-screen interface and associated with at least two targets from the character pane, the at least two targets corresponding to at least two of the plurality of characters, numerals, or punctuation marks;

selecting a most likely character, numeral, or punctuation mark among the at least two characters, numerals, or punctuation marks using one or more target models, the one or more target models modeling distributions of coordinate touch-locations on the touch-screen interface of one or more historic input events corresponding to at least one of the at least two targets among the plurality of targets;

displaying the most likely character on the touch-screen interface; and

using a set of swipe gestures to perform one or more functions on the touch-screen interface.

2. The non-transitory computer readable medium as recited in claim 1, further comprising the step of changing the character pane in response to a first set of swipe gestures.

3. The non-transitory computer readable medium as recited in claim 2, wherein a swipe gesture among the first set of swipe gestures changes the character pane from all capitals to all lowercase.

4. The non-transitory computer readable medium as recited in claim 2, wherein a swipe gesture among the first set of swipe gestures changes the character pane from letters to numbers and symbols.

5. The non-transitory computer readable medium as recited in claim 2, wherein a swipe gesture among the first set of gestures changes a font of the character pane.

6. The non-transitory computer readable medium as recited in claim 2, wherein a swipe gesture among the first set of swipe gestures changes the character pane to special characters.

7. The non-transitory computer readable medium as recited in claim 1, wherein each model among the one or more target models comprises a probability distribution, the probability distribution including a 2D Gaussian distribution, a Laplace distribution, and a Gamma distribution.

8. The non-transitory computer readable medium as recited in claim 1, wherein each model among the one or more target models is associated with an individual target from the character pane.

9. The non-transitory computer readable medium as recited in claim 1, wherein the one or more target models are selected from a plurality of target models based on one or more conditions.

10. The non-transitory computer readable medium as recited in claim 9, wherein the one or more conditions include an application type associated with the interface, a screen size, and a layout of the character pane.

11. The non-transitory computer readable medium as recited in claim 1, further comprising the step of using a swipe gesture to change the one or more target models to one or more alternative target models.

12. The non-transitory computer readable medium as recited in claim 1, wherein the step of selecting the most likely character includes the steps of:

generating a set of likely targets based on the current touch-input event by using the one or more target models, the set of likely targets including a set of probability values indicating a probability that a likely target among the set of likely targets was intended to be selected based on the current touch-input event; and

selecting a target with a highest probability value among the set of likely targets as the most likely character.

13. The non-transitory computer readable medium as recited in claim 1, further comprising the steps of:

generating one or more text predictions with a text prediction engine; and

displaying on the touch-screen interface the one or more text predictions generated with the text prediction engine.

14. The non-transitory computer readable medium as recited in claim 13, further comprising the steps of:

changing the one or more text predictions displayed on the touch-screen interface to alternative text predictions in response to a second set of swipe gestures; and

browsing the alternative text predictions using a third set of swipe gestures.

15. The non-transitory computer readable medium as recited in claim 1, wherein a swipe gesture among the set of swipe gestures performs an undo operation.

16. The non-transitory computer readable medium as recited in claim 1, wherein a swipe gesture among the set of swipe gestures terminates text entry.

17. The non-transitory computer readable medium as recited in claim 1, wherein a swipe gesture among the set of swipe gestures is a linear gesture.

18. The non-transitory computer readable medium as recited in claim 1, wherein a swipe gesture among the set of swipe gestures is a non-linear gesture.

19. A non-transitory computer readable medium embodying a program of instructions executable by a small screen device to perform program steps for operating a touch-screen interface on the small screen device, the program steps comprising:

displaying a typing pane to display inputted text on the touch-screen interface;

displaying a QWERTY-style character pane on the touch-screen interface defining a plurality of targets corresponding to a plurality of characters of the character pane;

accepting a current touch-input event, the current touch-input event corresponding to a coordinate location on the touch-screen interface and associated with at least two targets from the character pane;

communicating the current touch-input event to a set of target models, each target model of the set of target models modeling distributions of coordinate touch-locations on the touch-screen interface of historic input events corresponding to at least one of the at least two targets among the plurality of targets;

using the set of target models to generate a set of likely targets and a set of likely probabilities associated with the set of likely targets, a probability among the set of likely probabilities indicating a likelihood that a likely target among the set of likely targets was intended to be selected based on the current touch-input event;

generating a set of word predictions based on the set of likely targets using a text prediction engine;

displaying the set of word predictions on a prediction pane displayed on the touch-screen interface; and

using a set of swipe gestures to perform one or more functions on the touch-screen interface.

20. The non-transitory computer readable medium as recited in claim 19, wherein the typing pane is positioned on a top portion of the interface, wherein the prediction pane is positioned below the typing pane, and wherein the QWERTY-style character pane is positioned below the prediction pane.

21. The non-transitory computer readable medium as recited in claim 19, wherein each model among the one or more target models comprises a probability distribution the probability distribution including a 2D Gaussian distribution, a Laplace distribution, and a Gamma distribution.

22. The non-transitory computer readable medium as recited in claim 19, further comprising the step of accepting a selection of a predicted word among the set of word predictions displayed on the prediction pane.

23. The non-transitory computer readable medium as recited in claim 22, further comprising the steps of:

creating a set of touch-input events associated with the predicted word; and

pairing a particular touch-input event among the set of touch-input events with a particular target among the character pane, wherein the predicted word provides a mapping between the particular touch-input event and the particular target;

adding the pairing between the particular touch-input event and the particular target to a training set; and

retraining a target model among the set of target models with the training set.

24. The non-transitory computer readable medium as recited in claim 19, wherein the set of word predictions include a set of probabilities associated with each word prediction among the set of word predictions, the set of probabilities indicating a likelihood that the user intended to type each word prediction.

25. The non-transitory computer readable medium as recited in claim 24, further comprising the step of removing one or more word predictions from the set of word predictions with a probability value less than a threshold.

26. The non-transitory computer readable medium as recited in claim 19, further comprising the steps of:

using a swipe gesture to query the text prediction engine to generate additional text predictions; and

displaying the additional text predictions on the prediction pane.

27. A method for enabling input of text into a small screen device, comprising the steps of:

displaying an interface without buttons on a touch-screen of the small screen device, the interface including a word prediction pane, a typing pane, and a QWERTY-style character pan, the QWERTY-style character pane defining a plurality of characters;

generating an initial word prediction without a current word input with a text prediction engine, the text prediction engine comprising a plurality of language models and configured to generate text predictions using the plurality of language models based on at least one of a context input or the current word input;

accepting a touch event associated with touch coordinates on the interface, the touch coordinates being associated with at least two characters from the plurality of characters;

predicting one or more candidate words, wherein the prediction is based on the touch coordinates, one or more previous text predictions and one or more target models, the one or more target models modelling distributions of touch coordinates on the interface of one or more historic input events corresponding to at least one of the two or more characters of the plurality of characters;

displaying the one or more candidate words on the prediction pane; and

enabling the user to control the interface using one or more swipe gestures.

28. The method as recited in claim 27, further comprising the step of enabling association of a first gesture with a first function of the interface.

29. The method as recited in claim 27, further comprising the step of enabling performance of a swipe gesture among the one or more swipe gestures to change the character pane from all capitals to all lowercase.

30. The method as recited in claim 27, further comprising the step of enabling performance of a swipe gesture among the one or more swipe gestures to change the character pane from letters to numbers and symbols.

31. The method as recited in claim 27, further comprising the step of enabling performance of a swipe gesture among the one or more swipe gestures to change a font of the character pane.

32. The method as recited in claim 27, further comprising the step of enabling performance of a swipe gesture among the one or more swipe gestures to change the character pane to special characters.

33. The method as recited in claim 27, further comprising the step of enabling performance of a swipe gesture among the one or more swipe gestures to undo one or more previous operations.

34. The method as recited in claim 27, further comprising the step of enabling performance of a swipe gesture among the one or more swipe gestures to terminate text entry.