Method for controlling a virtual keyboard from a touchpad of a computerized device

• US 9,891,820 B2
• Filed: 07/25/2014
• Issued: 02/13/2018
• Est. Priority Date: 04/23/2010
• Status: Expired due to Fees

• Current Assignee: Handscape Inc., A Delaware Corporation
• Original Assignee: Handscape Inc., A Delaware Corporation
• Inventors: Luo, Tong
• Primary Examiner(s): Dasgupta, Shourjo
• Application Number:
  US14/341,326
• Publication Number:
  US 20140337786A1
• Time in Patent Office: 1,299 Days
• Field of Search: 715773, 345173-174, 345441, 345649, 345359-360
• CPC Class Codes: G06F 2203/04104 Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/0416 Control or interface arrangements specially adapted for digitisers
• CPC Class Codes: G06F 3/04186 Touch location disambiguation
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: H04L 69/329 in the application layer [OSI layer 7]

Abstract

A method for controlling a virtual keyboard on a display screen of a computerized system includes obtaining data from a touchpad. The data is associated with the location and movement of a finger and/or hand of a user when the user operates the computerized system using the touchpad. The method includes communicating the data from the touchpad to the computerized device, the touchpad being located in a location that is different from the location of the display screen. The method further includes analyzing the data in accordance with a model of a human hand, and assigning the data to at least one of a plurality of fingers of the model. The method also includes generating a virtual keyboard on the display screen and repositioning the virtual keyboard according to either a verbal command from the user or a user input from the touchpad.

Claims

1. A method for controlling a virtual keyboard on a display screen of a computerized device comprising a touchpad, processor and display screen, said method comprising:

obtaining data from a touchpad, the data being associated with the location and movement of at least one of a finger and hand of a user, when the user operates said computerized device using the touchpad with said at least one of a finger and hand positioned at an arbitrary location and angle with respect to said touchpad, the data not being associated with an image of the finger of the user from an image sensor;

communicating the data from the touchpad to said computerized device, the touchpad being located in a location that is different from the location of the display screen;

analyzing the data in accordance with a software model of a human hand;

said model of a human hand being configured to use mathematical angle rotation and coordinate transformations to model the positions of said user'"'"'s finger and hands that are positioned at arbitrary locations and angles with respect to said touchpad, and assigning the data to at least one of a plurality of fingers of the model;

generating a virtual keyboard on the display screen; and

repositioning the virtual keyboard according to said model of a human hand and said user input from the touchpad.

2. The method of claim 1, further comprising:

computing a graphical representation of at least one finger of the user in accordance with said model of the human hand; and

displaying the graphical representation on the display screen.

3. The method of claim 1, wherein generating a virtual keyboard on the display screen comprises:

tracking position and orientation of the user'"'"'s hand;

taking a time average of the position and orientation of the user'"'"'s hand over a first time period; and

generating a virtual keyboard on the display screen at a position and orientation based on the time average of the position and orientation of the user'"'"'s hand.

4. The method of claim 3, wherein the first time period is between 0.1 seconds to two minutes.

5. The method of claim 1, further comprising:

prior to repositioning the virtual keyboard, receiving a command from the user to enable said computerized device for repositioning the virtual keyboard.

6. The method of claim 1, wherein repositioning the virtual keyboard comprises receiving a multi-touch command comprising multiple touch pad inputs at the same time from the touchpad.

7. The method of claim 1, wherein said touchpad is located on a side of the computerized device that is behind the side of the computerized device that holds the display screen.

8. The method of claim 1, wherein said touchpad is located on a same side of the computerized device as the display screen, but at a different location from that of the display screen.

9. The method of claim 1, wherein said touchpad is configured to detect the proximity of the user'"'"'s fingers above the surface of the touch pad.

10. The method of claim 1, wherein the virtual keyboard is a graphical display of a keyboard including one or more data entry location, or a hyperlink.

11. The method of claim 1, wherein the virtual keyboard is a graphical display of at least part of a QWERTY keyboard.

12. The method of claim 1, wherein the virtual keyboard is a graphical display of number entry keyboard.

13. The method of claim 1, wherein repositioning the virtual keyboard comprises a verbal command including at least one of an angle of rotation and a distance of movement.

14. The method of claim 1, further comprising:

before repositioning the virtual keyboard, receiving a command from the user to enable said computerized device for keyboard repositioning.

15. The method of claim 14, wherein said command from the user comprises one or more of key press, voice command, virtual key selection, or touch pad input.

16. The method of claim 1, wherein repositioning the virtual keyboard comprises one or more of rotating the virtual keyboard, stretching and shrinking the virtual keyboard, and changing key spacing.

17. The method of claim 1, further repositioning the virtual keyboard according to a verbal command from the user.

18. A method for controlling a virtual keyboard on a display screen of a computerized device comprising a touchpad, processor and display screen, the method comprising:

obtaining data from a touchpad that is located on an opposite side of said computerized device as the display screen, the data being associated with the location and movement of at least one of a finger and hand of a user, when the user operates the computerized device using the touchpad with said at least one of a finger and hand positioned at an arbitrary location and angle with respect to said touchpad, the data not being associated with an image of the finger of the user from an image sensor;

communicating the data from the touchpad to said computerized device;

analyzing the data in accordance with a software model of a human hand;

said model of a human hand being configured to use mathematical angle rotation and coordinate transformations to model the positions of said user'"'"'s finger and hands that are positioned at arbitrary locations and angles with respect to said touchpad, and assigning the data to at least one of a plurality of fingers of the model;

computing a graphical representation of at least one finger of the user in accordance with the model of the human hand;

displaying the graphical representation of at least one finger of the user on the display screen;

using said model of a human hand to track the position and orientation of the user'"'"'s hand;

taking a time average of the position and orientation of the user'"'"'s hand over a first time period;

generating a virtual keyboard on the display screen at a position and orientation according to the time average of the position and orientation of the user'"'"'s hand;

receiving a command from the user to enable said computerized device for repositioning the virtual keyboard; and

repositioning the virtual keyboard according to either a verbal command from the user or a user input from the touchpad.

19. The method of claim 18, wherein the first time period is between 0.1 seconds to two minutes.

20. The method of claim 18, wherein repositioning the virtual keyboard comprises receiving a multi-touch command comprising multiple touch pad inputs at the same time from the touchpad.

21. The method of claim 18, wherein said command from the user comprises one or more of key press, voice command, virtual key selection, or touch pad input.