Method using a predicted finger location above a touchpad for controlling a computerized system

• US 9,542,032 B2
• Filed: 05/28/2014
• Issued: 01/10/2017
• Est. Priority Date: 04/23/2010
• Status: Expired due to Fees

• Current Assignee: Handscape Inc., A Delaware Corporation
• Original Assignee: Handscape Inc., A Delaware Corporation
• Inventors: Luo, Tong
• Primary Examiner(s): Edwards, Carolyn R
• Assistant Examiner(s): Gyawali, Bipin
• Application Number:
  US14/289,260
• Publication Number:
  US 20140267121A1
• Time in Patent Office: 958 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 1/1626 with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/041661 using detection at multiple resolutions, e.g. coarse and fine scanning; using detection within a limited area, e.g. object tracking window
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus

Abstract

A method for controlling an input from a user to a computerized system including a touchpad is presented. The method includes obtaining data from the touchpad. The data is associated with the location and movement of fingers and/or a hand of the user and not associated with an image of a first finger from an image sensor. The method further includes communicating the data from the touchpad to the computerized system and analyzing the data in accordance with a model of a human hand. The method further includes predicting the location of the first finger in accordance with the analyzed data and the location of at least a second finger and/or the hand. The second finger and/or the hand is touching the touchpad. The method further includes determining, using the model, that the first finger is positioned above but not touching a control region of the touchpad.

Claims

1. A method for controlling an input from a user to a computerized system comprising a touchpad, at least one processor, and at least one display screen, the method comprising:

using said at least one processor to display a graphical representation of a plurality of different control regions at different location coordinates on said display screen, said control regions responsive to touch input from said touchpad;

obtaining data from the touchpad, the data being associated with the location and movement of a plurality of fingers and/or a hand of the user at an arbitrary location and angle with respect to the touchpad, and not associated with an image of the plurality of fingers from an image sensor, when the user operates the computerized system using the touchpad;

communicating the data from the touchpad to the computerized system, and using said at least one processor to map said data from the touchpad to different location coordinates on said display screen, said display screen being in a different location from the location of said touchpad;

using said at least one processor to analyze the data in accordance with a software model of a human hand and assigning said data to at least a specific first finger and a second finger on said model;

using said at least one processor to predict, even when the specific first finger is not in contact with said touchpad, the location coordinates of the specific first finger in accordance with the analyzed data and touchpad contact location coordinates of at least a second finger of the plurality of fingers different than the specific first finger, wherein at least the second finger and/or the hand is touching the touchpad; and

determining, using said software model of a human hand, which of said plurality of fingers is the specific first finger, and that the specific first finger is positioned above but not touching a first control region that has location coordinates that are distinct from other control regions of the screen and associated mapped touchpad coordinates, thus distinguishing that said specific first finger is hovering specifically over the location coordinates of said first control region and not over location coordinates of other control regions.

2. The method of claim 1, further comprising:

assigning the data to at least one of a plurality of fingers of said model of the human hand;

computing a first graphical representation of the specific first finger of the user in accordance with said model of the human hand, wherein said graphical representation is responsive to which finger of said user'"'"'s hand is said specific first finger; and

displaying the first graphical representation on said at least one display screen of the computerized system.

3. The method of claim 2, wherein analyzing the data and assigning the data includes determining if the data may be assigned to one or more finger or hand portions on said model of the human hand according to a mathematical rotation operation upon at least a portion of the data.

4. The method of claim 1, further comprising:

generating an audible signal when the specific first finger of the user is positioned above but not touching the first control region.

5. The method of claim 1, further comprising:

determining, using said model of the human hand, that a plurality of fingers of the user are positioned above but not touching a first control region of the touchpad.

6. The method of claim 5, further comprising:

determining that the plurality of fingers are subsequently touching the first control region in accordance with the data and said model of the human hand after the plurality of fingers of the user are positioned above but not touching the first control region.

7. The method of claim 6, further comprising:

determining a motion of the specific first finger in relation to a motion of a third finger different than the specific first finger and second finger; and

assigning a command to control the computerized system in accordance with the determined motion.

8. The method of claim 1, further comprising:

prior to distinguishing that the specific first finger is hovering over the first control region, determining that the specific first finger is initially touching the first control region in accordance with the data and said model of the human hand before the specific first finger is positioned above but not touching the first control region.

9. The method of claim 8, further comprising:

generating a first graphical representation on a display screen of the computerized system, the first graphical representation being associated with the first control region;

displaying the first graphical representation with a first different appearance when the specific first finger is initially touching the first control region; and

displaying the first graphical representation with a second different appearance different than the first different appearance when the specific first finger is positioned above but not touching the first control region.

10. The method of claim 9, wherein the second different appearance includes a difference from the first different appearance and said first graphical representation, the difference being associated with at least one of a size, a color, a position, a shape, or a display type.

11. The method of claim 9, wherein the displaying the first graphical representation with a second different appearance occurs during a predetermined time window.

12. The method of claim 1, further comprising:

prior to distinguishing that the specific first finger is hovering over the first control region, determining that the specific first finger is subsequently touching the first control region in accordance with the data and said model of the human hand after the specific first finger is positioned above but not touching the first control region.

13. The method of claim 12, further comprising:

generating a first graphical representation on a display screen of the computerized system, the first graphical representation being associated with the first control region;

displaying the first graphical representation with a first different appearance when the specific first finger is positioned above but not touching the first control region; and

displaying the first graphical representation with a second different appearance different than the first different appearance and the first different appearance when the specific first finger is subsequently touching the first control region.

14. The method of claim 13, wherein the second different appearance includes a difference from the first different appearance and the first appearance, the difference associated with at least one of a size, a color, a position, a shape, or a display type.

15. The method of claim 13, wherein both the touchpad and the display screen are mounted on a same device, and wherein the display screen is mounted on a front location of said device, and the touchpad is located in a back location of said device.

16. The method of claim 13, wherein both the touchpad and the display screen are mounted on a same device, and wherein the touchpad is located in a location on said same device that is on substantially a same side as the location of the display screen on said same device.

17. The method of claim 12, further comprising:

storing a record that the specific first finger is subsequently touching the first control region.

18. The method of claim 12, further comprising:

storing a record that the specific first finger is subsequently touching the first control region during a predetermined time window.

19. The method of claim 12, further comprising:

determining that the specific first finger is moving or sliding on the first control region after subsequently touching the first control region during a predetermined time window; and

storing a record of the moving or sliding of the specific first finger.

20. The method of claim 12, further comprising:

determining that the specific first finger is moving or sliding on the first control region after subsequently touching the first control region; and

storing a record of the moving or sliding of the specific first finger.

21. The method of claim 20, further comprising:

generating a first graphical representation of a control region on a display screen of the computerized system, the first graphical representation of a control region being associated with the first control region, wherein the first graphical representation of the first control region includes a length that is greater than a longitudinal length of a surface region of the specific first finger when contacting the touchpad.

22. The method of claim 1, wherein the computerized system is a computerized system in a vehicle, a handheld tablet or cell phone, or a wearable computerized device.

23. The method of claim 1, wherein the computerized system is a handheld computerized device, and the touchpad is located on a side of the handheld computerized device that is different from the side of the handheld computerized device that displays the display screen, the display screen being non-transparent.

24. The method of claim 1, further comprising:

storing the data over a plurality of time intervals to form a history of recent finger positions; and

determining an approximate fingertip location coordinates and fingertip identity of a fingertip that is not touching the touchpad at a present time interval according to the history and data from the touchpad obtained at the present time interval.

25. The method of claim 1, further comprising:

distinguishing between a plurality of distances between the specific first finger of the user and a surface of the touchpad; and

detecting the location coordinates and movement of a finger and/or hand of the user to include in the data when the distance from the specific first finger of the user and the surface of the touchpad is greater than zero.

26. A method for controlling an input from a user to a computerized system comprising a touchpad, at least one processor, and at least one display screen, the method comprising:

using said at least one processor to display a graphical representation of a plurality of different control regions at different location coordinates on said display screen, said control regions responsive to touch input from said touchpad;

obtaining data from the touchpad, the data being associated with the location and movement of a plurality of fingers and/or a hand of the user at an arbitrary location and angle with respect to the touchpad, and not associated with an image of the plurality of fingers from an image sensor, when the user operates the computerized system using the touchpad;

communicating the data from the touchpad to the computerized system, and using said at least one processor to map said data from the touchpad to different location coordinates on said display screen, said display screen being in a different location from the location of said touchpad;

using said at least one processor to analyze the data in accordance with a software model of a human hand and assigning said data to at least a specific first finger and a second finger on said model;

wherein analyzing the data and assigning the data includes determining if the data may be assigned to one or more finger or hand portions on said model of the human hand according to a mathematical rotation operation upon at least a portion of the data;

storing the data over a plurality of time intervals to form a history of recent finger positions; and

determining an approximate fingertip location coordinates and fingertip identity of a fingertip that is not touching the touchpad at a present time interval according to the history and data from the touchpad obtained at the present time interval;

using said at least one processor to predict, even when a the specific first finger is not in contact with said touchpad, the location coordinates of the specific first finger in accordance with the analyzed data and touchpad contact location coordinates of at least a second finger of the plurality of fingers different than the specific first finger, wherein at least the second finger and/or the hand is touching the touchpad; and

determining, using said software model of a human hand, which of said plurality of fingers is the specific first finger, and that the specific first finger is positioned above but not touching a first control region that has location coordinates that are distinct from other control regions of the screen and associated mapped touchpad coordinates, thus distinguishing that said specific first finger is hovering specifically over the location coordinates of said first control region and not over location coordinates of other control regions.