Virtual keypad generator with learning capabilities

• US 8,300,023 B2
• Filed: 04/10/2009
• Issued: 10/30/2012
• Est. Priority Date: 04/10/2009
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Forutanpour, Babak, Abrahamson, Kurt W.
• Primary Examiner(s): Sherman, Stephen
• Application Number:
  US12/422,077
• Publication Number:
  US 20100259561A1
• Time in Patent Office: 1,299 Days
• Field of Search: 345168-172, 345173-178, 178 1801- 1809, 178/18.11, 178/20.01, 715/700, 715/866, 715/762, 715/764, 715/765, 715771-773
• US Class Current: 345/173
• CPC Class Codes: G06F 21/36 by graphic or iconic representation
• CPC Class Codes: G06F 3/0216 Arrangements for ergonomically adjusting the disposition of keys of a keyboard for keyboards integrated in a laptop computer G06F1/1664
• CPC Class Codes: G06F 3/0236 using selection techniques to select from displayed items
• CPC Class Codes: G06F 3/0237 using prediction or retrieval techniques
• CPC Class Codes: G06F 3/0238 Programmable keyboards key guide holders G06F3/0224
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus

Abstract

Methods and systems enable defining customized virtual keyboards on a touch sensitive surface, such as a touchscreen. Using learning algorithms, a computing device may learn the typing patterns of a user and “morph” the keys of a virtual keyboard into locations, sizes, and orientations that are comfortable for the user and which may reduce typing errors. A user may create a custom keypad layout by performing a series of key strokes on a touch sensitive surface. The computing device may correlate the touch locations to particular keys and generate keypad layout data that can be used to display the virtual keyboard and interpret touches on the touch sensitive surface. The computing device may monitor user typing activity and refine the keypad layout data based on detected or corrected typing errors. Keypad layout data may be exported to other devices to enable users to take their virtual keyboards with them.

Claims

1. A method for defining a keypad on a touch sensitive surface, comprising:

receiving a series of coordinates of a series of user touches on the touch sensitive surface;

correlating a plurality of keys with the received series of coordinates;

determining an average of received coordinates correlated with each of the plurality of keys, wherein determining the average of received coordinates correlated with each of the plurality of keys comprises;

calculating a variability of key-strike locations for each of the plurality of keys;

determining when the calculated variability of key-strike locations for each of the plurality of keys has plateaued; and

calculating an average of coordinate key strike locations for each of the plurality of keys once the variability of key strike locations has plateaued;

saving the calculated average coordinates for each of the plurality of keys in keypad layout data; and

generating an image of a virtual keypad using the keypad layout data.

2. The method of claim 1, further comprising:

prompting the user to enter a series of keystrokes,

wherein correlating a plurality of keys with the received series of coordinates comprises correlating the prompted series of keystrokes with the received coordinates.

3. The method of claim 1, further comprising:

determining a statistical envelope based upon received coordinates for each of the plurality of keys by calculating standard deviations about the calculated average coordinates for each of the plurality of keys; and

saving the statistical envelope for each of the plurality of keys in the keypad layout data.

4. The method of claim 3, wherein generating an image of the virtual keypad using the keypad layout data further comprises:

generating images of the plurality of keys in the virtual keypad in which a size of each key image is based upon the statistical envelope for each of the plurality of keys in the keypad layout data.

5. The method of claim 3, further comprising:

determining a first standard deviation envelope of key strike locations about the calculated average coordinates for each of the plurality of keys; and

interpreting a key strike as intended for a selected key in response to the key strike occurring within the first standard deviation envelope for the selected key.

6. The method of claim 5, further comprising:

determining a second standard deviation envelope of key strike locations about the calculated average coordinates for each of the plurality of keys;

determining when a key strike location is between and outside of respective first standard deviation envelopes for a first key and a second key; and

interpreting the key strike as intended for the first key in response to the key strike location being within the second standard deviation envelope of the first key and outside the second deviation envelop of the second key.

7. The method of claim 1, wherein generating an image of the virtual keypad using the keypad layout data comprises:

generating images of keys in the virtual keypad in locations based upon the average coordinates of each of the plurality of keys stored in the keypad layout data.

8. The method of claim 1, further comprising:

monitoring typing on the virtual keypad on the touch sensitive surface;

identifying an adjacent key typing error;

determining a correct key associated with the adjacent key typing error;

updating the average coordinate for the correct key based upon received coordinates for the typed keystroke; and

saving the updated average coordinate with the correct key in the keypad layout data.

9. The method of claim 8, wherein identifying an adjacent key typing error comprises:

recognizing a misspelled word; and

determining whether the misspelling involves a switch of two keys that are adjacent to each other in the virtual keypad.

10. The method of claim 8, wherein identifying an adjacent key typing error comprises:

recognizing a user correction of a letter associated with a key; and

determining whether the user correction involves a switch of two keys that are adjacent to each other in the virtual keypad.

11. The method of claim 8, wherein identifying an adjacent key typing error comprises:

prompting the user to type a specified phrase over a number of cycles;

recognizing a mistyped character based upon the specified phrase; and

determining whether the mistyped character involves a switch of two keys that are adjacent to each other in the virtual keypad.

12. The method of claim 1, wherein the keypad layout data is saved in a network accessible database, the method further comprising:

transmitting the keypad layout data to a computing device via the network,

wherein generating an image of a virtual keypad using the keypad layout data comprises generating the image of the virtual keypad on a touch sensitive surface coupled to the computing device.

13. The method of claim 12, further comprising:

receiving a user touch on the touch sensitive surface coupled to the computing device; and

determining a location on the touch sensitive surface for display of the virtual keypad based upon the received user touch on the touch sensitive surface,

wherein generating an image of a virtual keypad using the keypad layout data comprises generating the image at the determined location on the touch sensitive surface for display.

14. The method of claim 1, further comprising:

receiving a user input to select and move a key relative to another key; and

adjusting layout dimensions of the generated image of the virtual keypad based upon the user input and generating another image of the virtual keypad.

15. The method of claim 1, wherein determining the variability of key-strike location for each key further comprises:

determining key-strike speed; and

determining a key-strike error rate.

16. The method of claim 1, further comprising:

prompting the user to rest palms on the touch sensitive surface; and

distinguishing keystrokes from the user resting palms on the touch sensitive surface.

17. A computing device, comprising:

a processor;

memory coupled to the processor; and

a touch sensitive surface coupled to the processor,

wherein the processor is configured with processor-executable instructions to perform processes comprising;

receiving a series of coordinates of a series of user touches on the touch sensitive surface;

correlating a plurality of keys with the received series of coordinates;

determining an average of received coordinates correlated with each of the plurality of keys, wherein determining the average of received coordinates correlated with each of the plurality of keys comprises;

calculating a variability of key-strike locations for each of the plurality of keys;

determining when the calculated variability of key-strike locations for each of the plurality of keys has plateaued; and

calculating an average of coordinate key strike locations for each of the plurality of keys once the variability of key strike locations has plateaued;

saving the calculated average coordinates for each of the plurality of keys in keypad layout data; and

generating an image of a virtual keypad using the keypad layout data.

18. The computing device of claim 17, wherein the processor is configured with processor-executable instructions to perform further processes comprising:

prompting the user to enter a series of keystrokes,

wherein correlating a plurality of keys with the received series of coordinates comprises correlating the prompted series of keystrokes with the received coordinates.

19. The computing device of claim 17, wherein the processor is configured with processor-executable instructions to perform further processes comprising:

determining a statistical envelope based upon received coordinates for each of the plurality of keys by calculating standard deviations about the calculated average coordinates for each of the plurality of keys; and

saving the statistical envelope for each of the plurality of keys in the keypad layout data.

20. The computing device of claim 19, wherein the processor is configured with processor-executable instructions such that generating an image of the virtual keypad using the keypad layout data further comprises:

generating images of the plurality of keys in the virtual keypad in which a size of each key image is based upon the statistical envelope for each of the plurality of keys in the keypad layout data.

21. The computing device of claim 17, wherein the processor is configured with processor-executable instructions such that generating an image of the virtual keypad using the keypad layout data comprises:

generating images of keys in the virtual keypad in locations based upon the average coordinates of each of the plurality of keys stored in the keypad layout data.

22. The computing device of claim 17, wherein the processor is configured with processor-executable instructions to perform further processes comprising:

monitoring typing on the virtual keypad on the touch sensitive surface;

identifying an adjacent key typing error;

determining a correct key associated with the adjacent key typing error;

updating the average coordinate for the correct key based upon received coordinates for the typed keystroke; and

saving the updated average coordinate with the correct key in the keypad layout data.

23. The computing device of claim 22, wherein the processor is configured with processor-executable instructions such that identifying an adjacent key typing error comprises:

recognizing a misspelled word; and

determining whether the misspelling involves a switch of two keys that are adjacent to each other in the virtual keypad.

24. The computing device of claim 22, wherein the processor is configured with processor-executable instructions such that identifying an adjacent key typing error comprises:

recognizing a user correction of a letter associated with a key; and

determining whether the user correction involves a switch of two keys that are adjacent to each other in the virtual keypad.

25. The computing device of claim 17, wherein the keypad layout data is saved in a network accessible database, wherein the processor is configured with processor-executable instructions to perform further processes comprising:

receiving the keypad layout data from the network accessible database via the network.

26. The computing device of claim 25, wherein the processor is configured with processor-executable instructions to perform further processes comprising:

receiving a user touch on the touch sensitive surface coupled to the computing device;

determining a location on the touch sensitive surface for display of the virtual keypad based upon the received user touch on the touch sensitive surface; and

generating the image of the virtual keypad using the keypad layout data at the determined location on the touch sensitive surface for display.

27. The computing device of claim 17,

wherein the processor is configured with processor-executable instructions to perform processes further comprising:

receiving a user input to select and move a key relative to another key; and

adjusting layout dimensions of the generated image of the virtual keypad based upon the user input and generating another image of the virtual keypad.

28. A computing device coupled to a touch sensitive surface, comprising:

means for receiving a series of coordinates of a series of user touches on the touch sensitive surface;

means for correlating a plurality of keys with the received series of coordinates;

means for determining an average of received coordinates correlated with each of the plurality of keys, wherein the means for determining the average of received coordinates correlated with each of the plurality of keys comprises;

means for calculating a variability of key-strike locations for each of the plurality of keys;

means for determining when the calculated variability of key-strike locations for each of the plurality of keys has plateaued; and

means for calculating an average of coordinate key strike locations for each of the plurality of keys once the variability of key strike locations has plateaued;

means for saving the calculated average coordinates for each of the plurality of keys in keypad layout data; and

means for generating an image of a virtual keypad using the keypad layout data.

29. The computing device of claim 28, further comprising:

means for prompting the user to enter a series of keystrokes,

wherein means for correlating a plurality of keys with the received series of coordinates comprises means for correlating the prompted series of keystrokes with the received coordinates.

30. The computing device of claim 28, further comprising:

means for determining a statistical envelope based upon received coordinates for each of the plurality of keys by calculating standard deviations about the calculated average coordinates for each of the plurality of keys; and

means for saving the statistical envelope for each of the plurality of keys in the keypad layout data.

31. The computing device of claim 30, wherein means for generating an image of the virtual keypad using the keypad layout data further comprises:

means for generating images of the plurality of keys in the virtual keypad in which a size of each key image is based upon the statistical envelope for each of the plurality of keys in the keypad layout data.

32. The computing device of claim 28, wherein means for generating an image of the virtual keypad using the keypad layout data comprises:

means for generating images of keys in the virtual keypad in locations based upon the average coordinates of each of the plurality of keys stored in the keypad layout data.

33. The computing device of claim 28, further comprising:

means for monitoring typing on the virtual keypad on the touch sensitive surface;

means for identifying an adjacent key typing error;

means for determining a correct key associated with the adjacent key typing error;

means for updating the average coordinate for the correct key based upon received coordinates for the typed keystroke; and

means for saving the updated average coordinate with the correct key in the keypad layout data.

34. The computing device of claim 33, wherein means for identifying an adjacent key typing error comprises:

means for recognizing a misspelled word; and

means for determining whether the misspelling involves a switch of two keys that are adjacent to each other in the virtual keypad.

35. The computing device of claim 33, wherein means for identifying an adjacent key typing error comprises:

means for recognizing a user correction of a letter associated with a key; and

means for determining whether the user correction involves a switch of two keys that are adjacent to each other in the virtual keypad.

36. The computing device of claim 28, wherein the keypad layout data is saved in a network accessible database, the computing device further comprising:

means for transmitting the keypad layout data to a computing device via the network,

wherein means for generating an image of a virtual keypad using the keypad layout data comprises means for generating the image of the virtual keypad on a touch sensitive surface coupled to the computing device.

37. The computing device of claim 36, further comprising:

means for receiving a user touch on the touch sensitive surface coupled to the computing device; and

means for determining a location on the touch sensitive surface for display of the virtual keypad based upon the received user touch on the touch sensitive surface,

wherein means for generating an image of a virtual keypad using the keypad layout data comprises means for generating the image at the determined location on the touch sensitive surface for display.

38. The computing device of claim 28, further comprising:

means for receiving a user input to select and move a key relative to another key; and

means for adjusting layout dimensions of the generated image of the virtual keypad based upon the user input and generating another image of the virtual keypad.

48. The non-transitory computer-readable medium of claim 37, wherein the stored processor-executable instructions are configured to cause the processor to perform operations further comprising:

receiving a user touch on the touch sensitive surface coupled to the computing device;

determining a location on the touch sensitive surface for display of the virtual keypad based upon the received user touch on the touch sensitive surface;

wherein the stored processor-executable instructions configured to cause the processor to generate an image of the virtual keypad using the keypad layout data are configured to cause the processor to perform operations;

generating the image at the determined location on the touch sensitive surface for display.

39. A non-transitory, computer-readable medium having stored thereon processor-executable instructions configured to cause a processor to perform operations, comprising:

receiving a series of coordinates of a series of user touches on the touch sensitive surface;

correlating a plurality of keys with the received series of coordinates;

determining an average of received coordinates correlated with each of the plurality of keys, wherein determining the average of received coordinates correlated with each of the plurality of keys comprises;

calculating a variability of key-strike locations for each of the plurality of keys;

determining when the calculated variability of key-strike locations for each of the plurality of keys has plateaued; and

calculating an average of coordinate key strike locations for each of the plurality of keys once the variability of key strike locations has plateaued;

saving the calculated average coordinates for each of the plurality of keys in keypad layout data; and

generating an image of a virtual keypad using the keypad layout data.

40. The non-transitory computer-readable medium of claim 39, wherein the stored processor-executable instructions are configured to cause the processor to perform operations further comprising:

prompting the user to enter a series of keystrokes, correlating the prompted series of keystrokes with the received coordinates.

41. The non-transitory computer-readable medium of claim 39, wherein the stored processor-executable instructions are configured to cause the processor to perform operations further comprising:

determining a statistical envelope based upon received coordinates for each of the plurality of keys by calculating standard deviations about the calculated average coordinates for each of the plurality of keys; and

saving the statistical envelope for each of the plurality of keys in the keypad layout data.

42. The non-transitory computer-readable medium of claim 41, wherein the stored processor-executable instructions configured to cause the processor to generate an image of the virtual keypad using the keypad layout data using the keypad layout data cause the processor to perform operations further comprising:

generating images of the plurality of keys in the virtual keypad in which a size of each key image is based upon the statistical envelope for each of the plurality of keys in the keypad layout data.

43. The non-transitory computer-readable medium of claim 39, wherein the stored processor-executable instructions configured to cause the processor to generate an image of the virtual keypad using the keypad layout data using the keypad layout data cause the processor to perform operations further comprising:

generating images of keys in the virtual keypad in locations based upon the average coordinates of each of the plurality of keys stored in the keypad layout data.

44. The non-transitory computer-readable medium of claim 39, wherein the stored processor-executable instructions are configured to cause the processor to perform operations further comprising:

monitoring typing on the virtual keypad on the touch sensitive surface;

identifying an adjacent key typing error;

determining a correct key associated with the adjacent key typing error;

updating the average coordinate for the correct key based upon received coordinates for the typed keystroke; and

saving the updated average coordinates with the correct key in the keypad layout data.

45. The non-transitory computer-readable medium of claim 44, wherein the stored processor-executable instructions configured to cause the processor to identify an adjacent key typing error cause the processor to perform operations further comprising:

recognizing a misspelled word; and

determining whether the misspelling involves a switch of two keys that are adjacent to each other in the virtual keypad.

46. The non-transitory computer-readable medium of claim 44, wherein the stored processor-executable instructions configured to cause the processor to identify an adjacent key typing error cause the processor to perform operations further comprising:

recognizing a user correction of a letter associated with a key; and

determining whether the user correction involves a switch of two keys that are adjacent to each other in the virtual keypad.

47. The non-transitory computer-readable medium of claim 39, wherein the keypad layout data is saved in a network accessible database, and wherein the stored processor-executable instructions cause the processor to perform operations further comprising:

transmitting the keypad layout data to a computing device via the network,

generating an image of a virtual keypad using the keypad layout data on a touch sensitive surface coupled to the computing device.

49. The non-transitory, computer-readable medium of claim 39, wherein the stored thereon processor-executable instructions are configured to cause a processor to perform operations further comprising:

receiving a user input to select and move a key relative to another key; and

adjusting layout dimensions of the generated image of the virtual keypad based upon the user input and generating another image of the virtual keypad.