Method for user input from the back panel of a handheld computerized device

• US 8,384,683 B2
• Filed: 05/04/2010
• Issued: 02/26/2013
• Est. Priority Date: 04/23/2010
• Status: Active Grant

• Current Assignee: Handscape Inc., A Delaware Corporation
• Original Assignee: Tong Luo
• Inventors: Luo, Tong
• Primary Examiner(s): Wang, Quan-Zhen
• Assistant Examiner(s): MA, CALVIN
• Application Number:
  US12/773,075
• Publication Number:
  US 20110261058A1
• Time in Patent Office: 1,029 Days
• Field of Search: 345/158, 345/163, 345/168, 345/173, 345/174, 345/178, 715/701, 715/702
• US Class Current: 345/173
• CPC Class Codes: G06F 1/1626 with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 1/1643 the display being associated to a digitizer, e.g. laptops that can be used as penpads details related to the relative motion of the display enclosure with respect to the body enclosure, e.g. to move between laptop and tablet PC configuration G06F1/1615
• CPC Class Codes: G06F 1/169 the I/O peripheral being an integrated pointing device, e.g. trackball in the palm rest area, mini-joystick integrated between keyboard keys, touch pads or touch stripes G06F1/1643 takes precedence; constructional details of pointing devices G06F3/033
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus

Abstract

The present invention relates to a handheld computerized device with a bit mapped display screen on the front panel, and a touchpad installed on the back panel. More particularly, the invention relates to a method and graphical user interface that enable the user to see the user'"'"'s finger position and motion from behind the device superimposed upon a virtual keyboard layout on the front panel. This allows the user to use a touchpad keypad on the back of the device to input keystrokes and mouse actions, and these will be reflected on the display screen on the front of the handheld computerized device as “virtual fingers” or equivalent.

Claims

1. A method of assisting user data entry into a handheld computerized device, said handheld computerized device comprising at least one touchpad, at least one graphics display screen, at least one processor, memory, and software, said method comprising:

displaying at least one data entry location on said at least one graphics display screen of said handheld computerized device;

obtaining data on the location and movement of the user'"'"'s fingers and/or hand using said touchpad, said user'"'"'s fingers and/or hand being positioned in an arbitrary manner with respect to said touchpad when contacting said touchpad;

said touchpad being located in a location of said handheld computerized device that is different from the location of said at least one display screen;

analyzing said data from said touchpad on the location and movement of said user'"'"'s fingers and/or hand according to a biomechanical and anatomical model of a human hand, and without using image sensor data of the user'"'"'s finger and/or hand assigning said data on the location and movement of said user'"'"'s fingers and/or hand to specific fingers on said biomechanical and anatomical model of said human hand, thereby making predictions as to the location of the user'"'"'s hand and fingers;

using said biomechanical and anatomical model of said human hand, and said predictions of the location of the user'"'"'s hand and fingers, to compute a graphical representation of at least said user'"'"'s fingers;

displaying said graphical representation of at least said user'"'"'s fingers on said at least one graphics display screen of said handheld computerized device;

wherein distances between said graphical representation of at least said user'"'"'s fingers on said at least one graphics display screen, and said at least one data entry location, give information to said user to facilitate said user to position said user'"'"'s fingers and/or hand on said at least one touchpad to enter user data into said at least one data entry location.

2. The method of claim 1, wherein said at least one data entry location is a graphical display of a keyboard or keypad comprised of a plurality of data entry locations, or wherein said at least one data entry location is a hyperlink.

3. The method of claim 1, wherein said at least one data entry location comprises a graphical display of a QUERTY keyboard or keypad, and said handheld computerized device lacks a physical QUERTY keyboard or keypad.

4. The method of claim 1, wherein said at least one data entry location comprises a graphical display of a number entry keyboard or keypad, and said handheld computerized device lacks a physical number entry keyboard or keypad.

5. The method of claim 1, wherein said touchpad is located on a side of said handheld computerized device that is behind the side of said handheld device that holds said at least one graphics display screen.

6. The method of claim 1, wherein said touchpad is capable of detecting the proximity of said user'"'"'s fingers above the surface of said touchpad.

7. The method of claim 6, wherein said touchpad is capable of distinguishing the distance between said user'"'"'s fingers and the surface of said touchpad when said distance is greater than zero.

8. The method of claim 1, in which said software comprises calibration software to enable the biomechanical and anatomical model of said human hand to be calibrated to the touchpad data to more accurately match the biomechanical and anatomical characteristics of said user'"'"'s fingers and/or hand;

and

wherein said biomedical and anatomical characteristics comprise at least the length of the user'"'"'s fingers, thereby allowing said model of said human hand to determine the length of said user'"'"'s fingers for a plurality of different hand configurations.

9. The method of claim 1, in which said at least one data entry location is highlighted on said at least one graphics display screen whenever said computerized device determines that at least one finger on said user'"'"'s hand has left the touchpad and the position and motion history of said at least one finger is consistent with a capability said at least one finger on said user'"'"'s hand to strike a position on said touchpad that is consistent with the location of said at least one data entry location on said at least one graphics display screen;

wherein said highlighting commences before said at least one finger has struck the next position on said keypad.

10. The method of claim 1, in which said software comprises predictive typing software configured to deduce the most probable letter or letters that the user is likely to type next, and the most probable finger or fingers that the user will use to type this most probable letter, and said computerized device utilizes keystroke predictions made by said predictive typing software, along with said data on the location and movement of said user'"'"'s fingers and/or hand using said touchpad, to compute the graphical representation of at least said user'"'"'s fingers.

11. A method of assisting user data entry into a handheld computerized device, said handheld computerized device comprising at least one touchpad, at least one graphics display screen, at least one processor, memory, and software, said method comprising:

displaying at least one data entry location on said at least one graphics display screen of said handheld computerized device;

wherein said at least one data entry location is a graphical display of a keyboard or keypad comprised of a plurality of data entry locations;

obtaining data on the location and movement of the user'"'"'s fingers and/or hand using said touchpad, said user'"'"'s fingers and/or hand being positioned in an arbitrary manner with respect to said touchpad when contacting said touchpad;

said touchpad being located on a side of said handheld computerized device that is behind the side of said handheld device that holds said at least one graphics display screen;

analyzing said data from said touchpad on the location and movement of said user'"'"'s fingers and/or hand according to a biomechanical and anatomical model of a human hand, and without using image sensor data of the user'"'"'s finger and/or hand assigning said data on the location and movement of said user'"'"'s fingers and/or hand to specific fingers on said biomechanical and anatomical model of said human hand, thereby making predictions as to the location of the user'"'"'s hand and fingers;

using said biomechanical and anatomical model of said human hand, and said predictions of the location of the user'"'"'s hand and fingers, to compute a graphical representation of at least said user'"'"'s fingers;

displaying said graphical representation of at least said user'"'"'s fingers on said at least one graphics display screen of said handheld computerized device;

wherein distances between said graphical representation of at least said user'"'"'s fingers on said at least one graphics display screen, and said at least one data entry location, give information to said user to facilitate said user to position said user'"'"'s fingers and/or hand on said at least one touchpad to enter user data into said at least one data entry location and;

wherein said handheld computerized device lacks a physical QUERTY keyboard or keypad.

12. The method of claim 11, wherein the graphical representation of at least said user'"'"'s fingers on said at least one graphics display screen is semi-transparent so as to enable the graphical display underneath said user'"'"'s fingers to be observed;

and

using predictive typing software configured to deduce the most probable letter or letters that the user is likely to type next, and the most probable finger or fingers that the user will use to type this most probable letter, and said computerized device utilizes keystroke predictions made by said predictive typing software, along with said data on the location and movement of said user'"'"'s fingers and/or hand using said touchpad, to compute the graphical representation of at least said user'"'"'s fingers.

13. The method of claim 11, wherein said at least one data entry location is a graphical display of a semi-transparent keyboard or semi-transparent keypad, thus allowing details of a current application running on said graphics display to be visualized along with said semi-transparent keyboard or semi-transparent keypad.

14. The method of claim 11, wherein the information of finger motion of a plurality of fingers, including motion type (e.g, touch, move, and stroke, etc), and motion position, is passed to a virtual keyboard processor, wherein said virtual keyboard processor analyzes the finger motion, compares finger position with the known location of said keys, and decides which key was stroked and invokes the corresponding operation, wherein the virtual keyboard processor also updates a real-time finger position image on the front screen after each finger motion;

Wherein said finger position image is based, at least in part, on the length of the user'"'"'s fingers and the position of the user'"'"'s fingers as determined by said biomechanical and anatomical model of said human hand.

15. The method of claim 11, wherein the graphics display is on the front panel screen of said device, and wherein said area of said graphics display may occupy up to the whole front panel screen of said device, and wherein the content of said graphics display can be visualized underneath semi-transparent representations of said user'"'"'s fingers and semi-transparent representations of said keyboard or keypad.

16. The method of claim 11, wherein said graphical representation of at least said user'"'"'s fingers on said at least one graphics display screen is done by the steps of:

using said assignment of said data from said touchpad on the location and movement of said user'"'"'s fingers and/or hand to specific fingers on said biomechanical and anatomical model of said human hand to create a three dimensional model of the user'"'"'s hand and fingers in memory;

said three dimensional model being based at least in part on the length of said user'"'"'s fingers;

creating a two-dimensional projection of said three dimensional model of the user'"'"'s hand and fingers in memory, wherein said two-dimensional projection is on an imaginary plane that corresponds in both distance and orientation to said touchpad;

and using said two-dimensional projection on said imaginary plane to generate said graphical representation of at least said user'"'"'s fingers on said at least one graphics display screen.

17. The method of claim 11, wherein said graphical representation of at least said user'"'"'s fingers on said at least one graphics display screen is done by the steps of:

using said assignment of said data on the location and movement of said user'"'"'s fingers and/or hand to specific fingers on said biomechanical and anatomical model of said human hand to manipulate a two-dimensional model of the user'"'"'s hand and fingers in memory;

said two dimensional model being based at least in part on the length of said user'"'"'s fingers;

and using said two-dimensional model to generate said graphical representation of at least said user'"'"'s fingers on said at least one graphics display screen.

18. A method of assisting user data entry into a handheld computerized device, said handheld computerized device comprising at least one touchpad, at least one graphics display screen, at least one processor, memory, and software, said method comprising:

displaying at least one data entry location on said at least one graphics display screen of said handheld computerized device;

wherein said at least one data entry location is a graphical display of a keyboard or keypad comprised of a plurality of data entry locations;

obtaining data on the location and movement of the user'"'"'s fingers and/or hand using said touchpad, said user'"'"'s fingers and/or hand being positioned in an arbitrary manner with respect to said touchpad when contacting said touchpad;

said touchpad being located on a side of said handheld computerized device that is behind the side of said handheld device that holds said at least one graphics display screen;

analyzing said data from said touchpad on the location and movement of said user'"'"'s fingers and/or hand according to a biomechanical and anatomical model of a human hand, and without using image sensor data of the user'"'"'s finger and/or hand assigning said data on the location and movement of said user'"'"'s fingers and/or hand to specific fingers on said biomechanical and anatomical model of said human hand, thereby making predictions as to the location of the user'"'"'s hand and fingers;

using said biomechanical and anatomical model of said human hand, and said predictions of the location of the user'"'"'s hand and fingers, to compute a graphical representation of at least said user'"'"'s fingers;

displaying said graphical representation of at least said user'"'"'s fingers on said at least one graphics display screen of said handheld computerized device;

wherein said graphical representation of at least said user'"'"'s fingers on said at least one graphics display screen is done by the steps of;

using said assignment of said data on the location and movement of said user'"'"'s fingers and/or hand to specific fingers on said biomechanical and anatomical model of said human hand to create a three dimensional model of the user'"'"'s hand and fingers in memory;

creating a two-dimensional projection of said three dimensional model of the user'"'"'s hand and fingers in memory, wherein said two-dimensional projection is on an imaginary plane that corresponds in both distance and orientation to said touchpad;

and using said two-dimensional projection on said imaginary plane to generate said graphical representation of at least said user'"'"'s fingers on said at least one graphics display screen;

wherein distances between said graphical representation of at least said user'"'"'s fingers on said at least one graphics display screen, and said at least one data entry location, give information to said user to facilitate said user to position said user'"'"'s fingers and/or hand on said at least one touchpad to enter user data into said at least one data entry location and;

wherein said handheld computerized device lacks a physical QUERTY keyboard or keypad.

19. The method of claim 18, in which the graphical representation of at least said user'"'"'s fingers on said at least one graphics display screen is semi-transparent so as to enable the graphical display underneath said user'"'"'s fingers to be observed;

and

using predictive typing software configured to deduce the most probable letter or letters that the user is likely to type next, and the most probable finger or fingers that the user will use to type this most probable letter, and said computerized device utilizes keystroke predictions made by said predictive typing software, along with said data on the location and movement of said user'"'"'s fingers and/or hand using said touchpad, to compute the graphical representation of at least said user'"'"'s fingers.

20. The method of claim 18, in which said software comprises calibration software to enable the biomechanical and anatomical model of said human hand to be calibrated to the touchpad data to more accurately match the biomechanical and anatomical characteristics of said user'"'"'s fingers and/or hand;

and

wherein said biomedical and anatomical characteristics comprise at least the length of the user'"'"'s fingers, thereby allowing said model of said human hand to determine the length of said user'"'"'s fingers for a plurality of different hand configurations.