Virtual touchscreen keyboards

• US 9,041,654 B2
• Filed: 12/17/2012
• Issued: 05/26/2015
• Est. Priority Date: 07/18/2012
• Status: Active Grant

• Current Assignee: SAP SE
• Original Assignee: SAP SE
• Inventors: Ellard, Michael Patrick
• Primary Examiner(s): Pappas, Claire X
• Assistant Examiner(s): Trandai, Scott
• Application Number:
  US13/716,422
• Publication Number:
  US 20140168083A1
• Time in Patent Office: 890 Days
• Field of Search: 345/173, 345/168
• US Class Current: 345/168
• CPC Class Codes: G06F 3/0216 Arrangements for ergonomically adjusting the disposition of keys of a keyboard for keyboards integrated in a laptop computer G06F1/1664
• CPC Class Codes: G06F 3/0236 using selection techniques to select from displayed items
• CPC Class Codes: G06F 3/0238 Programmable keyboards key guide holders G06F3/0224
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus

Abstract

A dynamically updated virtual keyboard may be presented on a touchscreen of a user device by detecting a first hand position on the touchscreen. A virtual keyboard may then be displayed on the touchscreen, the virtual keyboard based on the first hand position. Characteristics of the virtual keyboard, such as size, position, orientation, and selection of the keys can also vary based on the first hand position and on other input features.

Claims

1. A method comprising:

detecting a first hand position on a touchscreen of an electronic device;

detecting a second hand position on the touchscreen;

detecting ten fingers on the touchscreen;

associating two fingers being furthest apart on the touchscreen with one pinkie of each of two different hands;

associating two fingers being next furthest apart on the touchscreen one ring finger of each of two different hands;

associating two fingers being next furthest apart on the touchscreen with one middle finger of each of two different hands;

associating two fingers being next furthest apart on the touchscreen with one index finger of each of two different hands;

associating two fingers being next furthest apart on the touchscreen with one thumb of each of two different hands;

deriving a line equation in form y=mx+b between the two pinkies, with y being a y-coordinate of the touchscreen and x being an x-coordinate of the touchscreen;

calculating a y-value corresponding to and using the line equation for an x-value of one of the thumbs;

when the y-value of the one of the thumbs is greater than the calculated y-value, associating a pinkie having a lowest x-value with a left hand; and

presenting a virtual keyboard on the touchscreen based on the first hand position, the second hand position, and the associations between fingers on the touchscreen and pinkies, ring fingers, middle fingers, index fingers, and thumbs.

2. The method of claim 1, further comprising detecting a size of a first hand and wherein a size of the virtual keyboard varies depending on the size of the first hand.

3. The method of claim 1, further comprising rotating at least a portion of the virtual keyboard based on the first hand position.

4. The method of claim 1, wherein the detecting includes detecting a plurality of fingers on the touchscreen and wherein the virtual keyboard is adjusted so that specific keys fall underneath particular fingers.

5. The method of claim 4, wherein the adjusting of the virtual keyboard so that specific keys fall underneath particular fingers include placing a home row of the keyboard along a line connecting the plurality of fingers.

6. The method of claim 1, wherein the virtual keyboard includes fewer keys than a corresponding physical keyboard, with keys on the corresponding physical keyboard missing from the virtual keyboard represented through gestures made by a user on keys present on the virtual keyboard.

7. The method of claim 4, further comprising:

detecting a home row of the virtual keyboard;

presenting the virtual keyboard so that the home row is oriented beneath the detected plurality of fingers.

8. The method of claim 1, further comprising:

extrapolating finger position based on the detected first hand position.

9. The method of claim 1, further comprising:

detecting typographical errors by a user and adjusting the virtual keyboard to reduce the typographical errors.

10. The method of claim 9, wherein the typographical errors are detected by tracking keys commonly pressed before pressing a backspace button.

11. The method of claim 1, further comprising discarding any touches when any of the following are true:

a distance from one hand'"'"'s ring finger to that hand'"'"'s middle finger is greater than a preset number of times that hand'"'"'s distance from the pinkie to ring finger;

a distance from one hand'"'"'s middle to index finger is greater than a preset number of times that hand'"'"'s distance from the pinkie to ring finger;

or

a distance from one hand'"'"'s index finger to thumb is less than a preset number of times that hand'"'"'s distance from the pinkie to ring finger.

12. A method comprising:

presenting a virtual keyboard to a client device for display on a touchscreen of the client device;

receiving, from the client device, a detected first hand position and a detected second hand position of a user on the touchscreen of the client device;

receiving, from the client device, a detected ten fingers on the touchscreen on the client device;

associating two fingers being furthest apart on the touchscreen with one pinkie of each of two different hands;

associating two fingers being next furthest apart on the touchscreen with one ring finger of each of two different hands;

associating two fingers being next furthest apart on the touchscreen with one middle finger of each of two different hands;

associating two fingers being next furthest apart on the touchscreen with one index finger of each of two different hands;

associating two fingers being next furthest apart on the touchscreen with one thumb of each of two different hands;

deriving a line equation in form y=mx+b between the two pinkies, with y being a y-coordinate of the touchscreen and x being an x-coordinate of the touchscreen;

calculating a y-value corresponding to and using the line equation for an x-value of one of the thumbs;

when the y-value of the one of the thumbs is greater than the calculated y-value, associating a pinkie having a lowest x-value with a left hand;

dynamically updating the virtual keyboard based on the detected first hand position of the user, the detected second hand position of the user, and the associations between fingers on the touchscreen and pinkies, ring fingers, middle fingers, index fingers, and thumbs; and

presenting the updated virtual keyboard to the client device for display on the touchscreen of the client device.

13. The method of claim 12, wherein the method is performed at a server.

14. A client device comprising:

a touchscreen;

a user input interpretation module comprising one or more processors and configured to detect a first hand position on the touchscreen, detect a second hand position on the touchscreen, and detect ten fingers on the touchscreen; and

a dynamic virtual keyboard updater configured to associate two fingers being furthest apart on the touchscreen with pinkie of each of two different hands, associate two fingers being next furthest apart on the touchscreen with one ring finger of each of two different hands, associate two fingers being next furthest apart on the touchscreen with one middle finger of each of two different hands, associate two fingers being next furthest apart on the touchscreen with one index finger of each of two different hands, associate two fingers being next furthest apart on the touchscreen with one thumb of each of two different hands, derive a line equation in form y=mx+b between the two pinkies, with y being a y-coordinate of the touchscreen and x being an x-coordinate of the touchscreen, calculate a y-value corresponding to and using the line equation for an x-value of one of the thumbs, when the y-value of the one of the thumbs is greater than the calculated y-value, associate a pinkie having a lowest x-value with a left hand, and present a virtual keyboard on the touchscreen based on the first hand position, the second hand position, and the associations between fingers on the touchscreen and pinkies, ring fingers, middle fingers, index fingers, and thumbs.

15. The client device of claim 14, wherein the client device is a mobile device.

16. The client device of claim 15, wherein the touchscreen is not large enough to accommodate a virtual keyboard placed where two hands of a user rest on the touchscreen.

17. A server comprising:

a processor;

a memory; and

a dynamic virtual keyboard updater configured to;

present a virtual keyboard to a client device for display on a touchscreen of the client device;

receive, from the client device, a detected first hand position of a user on the touchscreen of the client device;

receive, from the client device, a detected second hand position of a user on the touchscreen of the client device;

receive, from the client device, a detected ten fingers on the touchscreen on the client device;

associate two fingers being furthest apart on the touchscreen with one pinkie of each of two different hands;

associate two fingers being next furthest apart on the touchscreen with one ring finger of each of two different hands;

associate two fingers being next furthest apart on the touchscreen with one middle finger of each of two different hands;

associate two fingers being next furthest apart on the touchscreen with one index finger of each of two different hands;

associate two fingers being next furthest apart on the touchscreen with one thumb of each of two different hands;

deriving a line equation in form y=mx+b between the two pinkies, with y being a y-coordinate of the touchscreen and x being an x-coordinate of the touchscreen;

calculating a y-value corresponding to and using the line equation for an x-value of one of the thumbs;

when the y-value of the one of the thumbs is greater than the calculated y-value, associating a pinkie having a lowest x-value with a left hand;

dynamically update the virtual keyboard based on the detected first hand position of the user, the detected second hand position of the user, and the associations between fingers on the touchscreen and pinkies, ring fingers, middle fingers, index fingers, and thumbs;

present the updated virtual keyboard to the client device for display on the touchscreen of the client device.

18. The server of claim 17, wherein the server is a web server.

19. A non-transitory computer-readable storage medium comprising instructions that, when executed by at least one processor of a machine, cause the machine to perform operations comprising:

detecting a first hand position on a touchscreen of an electronic device;

detecting a second hand position on the touchscreen;

detecting ten fingers on the touchscreen;

associating two fingers being furthest apart on the touchscreen with one pinkie of each of two different hands;

associating two fingers being next furthest apart on the touchscreen one ring finger of each of two different hands;

associating two fingers being next furthest apart on the touchscreen with one middle finger of each of two different hands;

associating two fingers being next furthest apart on the touchscreen with one index finger of each of two different hands;

associating two fingers being next furthest apart on the touchscreen with one thumb of each of two different hands;

deriving a line equation in form y=mx+b between the two pinkies, with y being a y-coordinate of the touchscreen and x being an x-coordinate of the touchscreen;

calculating a y-value corresponding to and using the line equation for an x-value of one of the thumbs;

when the y-value of the one of the thumbs is greater than the calculated y-value, associating a pinkie having a lowest x-value with a left hand; and

presenting a virtual keyboard on the touchscreen based on the first hand position, the second hand position, and the associations between fingers on the touchscreen and pinkies, ring fingers, middle fingers, index fingers, and thumbs.