Method for user input from alternative touchpads of a computerized system

• US 9,430,147 B2
• Filed: 04/23/2014
• Issued: 08/30/2016
• Est. Priority Date: 04/23/2010
• Status: Active Grant

• Current Assignee: Handscape Inc., A Delaware Corporation
• Original Assignee: Handscape Inc., A Delaware Corporation
• Inventors: Luo, Tong
• Primary Examiner(s): Haley, Joseph
• Assistant Examiner(s): FRANK, EMILY J
• Application Number:
  US14/260,195
• Publication Number:
  US 20140313168A1
• Time in Patent Office: 860 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus

Abstract

A method includes obtaining data from a touchpad, the data being associated with the location and movement of a user'"'"'s finger and/or hand and not associated with an image of the user'"'"'s finger from an image sensor, when the user operates the computerized system using the touchpad. The method further includes communicating the data from the touchpad to the computerized device, the touchpad being located in a location that is different from the location of the display screen. The method further includes analyzing the data in accordance with a model of a human hand, and assigning the data to at least one of a multitude of fingers of the model, computing a graphical representation of at least one finger of the user in accordance with the model, and displaying the graphical representation on the display screen.

Claims

1. A method for displaying a virtual image of at least a portion of a hand of a user on a display screen of a computerized system, the method comprising:

obtaining data from a touchpad, the data being associated with the location and movement of a finger and/or hand of the user and not associated with an image of the finger of the user from an image sensor, when the user operates the computerized system using the touchpad;

communicating the data from the touchpad to the computerized device, the touchpad being located in a location that is different from the location of the display screen;

analyzing the data in accordance with a model of a human hand with at least one of adjustable hand size or orientation parameters, and assigning the data to at least one of a plurality of fingers of the model;

computing a graphical representation of at least one finger of the user in accordance with the model; and

displaying the graphical representation on the display screen.

2. The method of claim 1, wherein the model includes calibration information associated with an image of at least a portion of the hand of the user.

3. The method of claim 1, wherein the model includes finger length calibration information in accordance with pressing a portion of the hand of the user on the touchpad.

4. The method of claim 1 further comprising utilizing a best fit between said data from said touchpad and said model to predict the location of the finger and/or hand of the user.

5. The method of claim 1, wherein the touchpad is not physically connected to the computerized system, wherein the touchpad communicates the data using at least one of a wireless, optical, infrared, or wired communication link.

6. The method of claim 1, wherein the touchpad is physically connected to the computerized system, wherein the touchpad communicates the data using at least one of a wireless, optical, infrared, or wired communication link.

7. The method of claim 1, wherein the touchpad has either a flat or a curved surface.

8. The method of claim 1, wherein the computerized system is a handheld computerized device, a computerized system in a vehicle, or a wearable computerized device.

9. The method of claim 1, wherein the computerized system is a handheld computerized device, and the touchpad is located on a side of the handheld computerized device that is different from the side of the handheld computerized device that holds the display screen, the display screen being non-transparent.

10. The method of claim 1, wherein analyzing the data and assigning the data includes determining if the data may be assigned to one or more finger or hand portions on the model according to a rotation operation upon at least a portion of the data.

11. The method of claim 1, wherein analyzing the data and assigning the data comprises:

identifying an approximate location of at least one of a plurality of metacarpophalangeal joints of the hand;

and determining a location and orientation of a palm of the user according to an orientation of the at least one of the plurality of metacarpophalangeal joints.

12. The method of claim 11 further comprising:

determining a probable location and orientation of a plurality of fingers of the user according to the location and orientation of the palm.

13. The method of claim 11 further comprising refining a likely location of at least one of the plurality of metacarpophalangeal joints according to a finger width of the model and the location and orientation of the palm.

14. The method of claim 13 further comprising determining a likely location of at least one of the plurality of fingertips of the user according to a finger length of the model and the likely location of at least one of the plurality of metacarpophalangeal joints and the location and orientation of the palm.

15. The method of claim 13 further comprising:

correlating the data with the likely location of at least one of the plurality of fingertips;

determining that at least one finger is elevated above a surface of the touchpad when the data indicates that the at least one finger is not touching the touchpad in the likely fingertip location, and displaying the graphical representation so as to indicate an elevated finger;

and determining that at least one finger is contacting the surface of the touchpad when the data indicates that the at least one finger is touching the touchpad in the likely fingertip location, and displaying the graphical representation so as to indicate a contacting finger having a different graphical representation than the elevated finger.

16. The method of claim 1 further comprising:

storing the data over a plurality of time intervals to form a history of recent finger positions;

determining an approximate fingertip location or fingertip identity of a fingertip that is not touching the touchpad at a present time interval according to the history and data from the touchpad obtained at the present time interval.

17. The method of claim 1, wherein the model further comprises information pertaining to a range of relative hand and finger angles permissible between a first hand of the user and a second hand of the user;

and using the range of relative hand and finger angles to process the data to determine which of the data likely corresponds to the first hand, and which of the data likely corresponds to the second hand.

18. The method of claim 1 further comprising:

analyzing the data and assigning the data according to an iterative process that includes;

making tentative assignments between the data and a plurality of alternative possible locations of fingertip, metacarpophalangeal joint, or palm touch point, at one or more possible palm orientations;

and evaluates each tentative assignment according to a scoring algorithm;

selecting the tentative assignment with a highest score to use in computing the graphical representation.

19. The method of claim 1 further comprising obtaining a partial image of a hand of the user to supplement or refine the model;

analyzing the partial image to compute one or more user parameters of the model;

and replacing one or more default parameters of the model according to the one or more user parameters.

20. The method of claim 1 further comprising:

using said touchpad to distinguish a distance between the at least one finger of the user and a surface of the touchpad;

and detecting the location and movement of a finger and/or hand of the user to include in the data when the distance that said finger is hovering above said touchpad is greater than zero.

21. The method of claim 1, wherein analyzing the data and assigning the data comprises:

performing a best fit between the data and the model;

determining coordinates associated with at least one of a plurality of metacarpophalangeal joints of the hand and at least one of a plurality of touchpad touch points of the hand of the user;

calculating an angle between at least one of the plurality of metacarpophalangeal joints and either an x axis of the touchpad, or a line passing through a point on a palm of the model;

identifying and predicting the approximate location of the plurality of fingertips of the model using the angle to, and determining if the identified and predicted approximate locations of the plurality of fingertips substantially match a portion of the data.

22. The method of claim 21, wherein the line passing through a point on the palm of the model also passes substantially through a mid-line of a middle finger (finger 2) of the model.