Data input device for individuals with limited hand function

• US 6,763,320 B2
• Filed: 08/15/2002
• Issued: 07/13/2004
• Est. Priority Date: 08/15/2002
• Status: Active Grant

• Current Assignee: Wistron Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Kimble, Christopher John
• Primary Examiner(s): Hoff, Marc S.
• Assistant Examiner(s): Raymond, Edward
• Application Number:
  US10/219,559
• Publication Number:
  US 20040034505A1
• Time in Patent Office: 698 Days
• Field of Search: 702/182, 702/42, 702/47, 702/134, 702/138, 702/150, 702/159, 702/196, 702/172, 710/8, 710/12, 710/67, 710/73
• US Class Current: 702/182
• CPC Class Codes: G06F 3/014 Hand-worn input/output arrangements, e.g. data gloves
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883

Abstract

A simulated keyboard data input device for a user with limited hand function comprises a hand-worn glove having multiple sensors capable of detecting minute motion of the fingers, the minute voluntary motion being significantly less than typically required for keystrokes in a conventional keyboard. A sensor interpreter is individually configured to recognize a user'"'"'s finger movements as respective keys of a keyboard. The user “types” text on the simulated keyboard by repeating those combinations of minute finger motion of which the user is capable to produce keyboard characters from the sensor interpreter. Preferably, the sensor interpreter comprises a virtual keystroke detector and a key decoder, both of which are individually calibrated.

Claims

1. A data input apparatus, comprising:

a sensing device containing a plurality of sensors capable of sensing voluntary motion of parts of a human hand of a user; and

a sensor interpreter coupled to said plurality of sensors in said sensing device, said sensor interpreter being configurable to an individual user to recognize different combinations of arbitrary motion of parts of at least one hand of said individual user as activations of respective different keys of a keyboard without requiring said at least one hand to touch another object and without requiring said at least one hand to assume a pre-determined configuration with respect to said plurality of sensors.

2. The data input apparatus of claim 1, wherein said sensing device comprises at least one glove fitting said human hand of said user, said plurality of sensors being mounted in said at least one glove.

4. The data input apparatus of claim 2, wherein said sensors in said at least one glove are coupled via a cable to a portable sensor receiver attached to the user, said sensor receiver receiving data from said sensors, and communicating data to a digital computing device.

7. The data input apparatus of claim 4, wherein said sensor receiver communicates with said digital computing device over a transmission interface.

8. The data input apparatus of claim 1, wherein said different combinations of arbitrary motion of said at least one hand are of significantly less magnitude than hand motion typical for activating keys of a keyboard.

9. The data input apparatus of claim 1, wherein said sensor interpreter comprises at least one neural network.

10. The data input apparatus of claim 1, wherein said sensor interpreter comprises:

(a) a virtual keystroke detector which detects the occurrence of hand movements corresponding to virtual keystrokes by the user; and

(b) a key decoder which decodes information from said plurality of sensors coincident with the detection of virtual keystrokes by said virtual keystroke detector as said keys of a keyboard.

11. The data input apparatus of claim 10, wherein said virtual keystroke detector detects a virtual keystroke as a maximum of a pre-defined deflection measure during a window, said pre-defined deflection measure being a function of data from said plurality of sensors.

12. The data input apparatus of claim 11, wherein said window is configurable to said individual user.

13. The data input apparatus of claim 11, wherein said pre-defined deflection measure is of the formula:

3. A data input apparatus, comprising:

a sensing device containing a plurality of sensors capable of sensing voluntary motion of parts of a human hand of a user; and

a sensor interpreter coupled to said plurality of sensors in said sensing device, said sensor interpreter being configurable to an individual user to recognize different combinations of arbitrary motion of parts of at least one hand of said individual user as activations of respective different keys of a keyboard without requiring said at least one hand to touch another object;

wherein said sensing device comprises at least one glove fitting said human hand of said user, said plurality of sensors being mounted in said at least one glove;

wherein said plurality of sensors comprise a plurality of strain gauges having variable electrical resistances.

5. A data input apparatus, comprising:

a sensing device containing a plurality of sensors capable of sensing voluntary motion of parts of a human hand of a user; and

a sensor interpreter coupled to said plurality of sensors in said sensing device, said sensor interpreter being configurable to an individual user to recognize different combinations of arbitrary motion of parts of at least one hand of said individual user as activations of respective different keys of a keyboard without requiring said at least one hand to touch another object;

wherein said sensing device comprises at least one glove fining said human hand of said user, said plurality of sensors being mounted in said at least one glove;

wherein said sensors in said at least one glove are coupled via a cable to a portable sensor receiver attached to the user, said sensor receiver receiving data from said sensors, and communicating data to a digital computing device;

wherein at least a portion of said sensor interpreter is contained in said sensor receiver.

6. The data input apparatus of claim 5, wherein said sensor interpreter comprises:

(a) a virtual keystroke detector which detects the occurrence of hand movements corresponding to virtual keystrokes by the user, said virtual keystroke detector being contained in said sensor receiver; and

(b) a key decoder which decodes information from said plurality of sensors coincident with the detection of virtual keystrokes by said virtual keystroke detector as said keys of a keyboard, said key decoder being contained in said digital computing device.

14. A data input apparatus, comprising:

a sensing device containing a plurality of sensors capable of sensing voluntary motion of parts of a human hand of a user; and

a sensor interpreter coupled to said plurality of sensors in said sensing device, said sensor interpreter being configurable to an individual user to recognize different combinations of arbitrary motion of parts of at least one hand of said individual user as activations of respective different keys of a keyboard without requiring said at least one hand to touch another object;

wherein said sensor interpreter comprises;

(a) a virtual keystroke detector which detects the occurrence of hand movements corresponding to virtual keystrokes by the user; and

(b) a key decoder which decodes information from said plurality of sensors coincident with the detection of virtual keystrokes by said virtual keystroke detector as said keys of a keyboard;

wherein said key decoder comprises at least one neural network, said neural network being configurable to said individual user.

15. A method for inputting data to a digital device, comprising the steps of:

configuring said digital device to recognize different combinations of arbitrary motion of parts of at least one hand of an individual user as activations of respective different keys of a keyboard, at least some of said different combinations of arbitrary motion of said at least one hand neither touching any other object nor assuming any pre-determined configuration with respect to sensing apparatus for sensing said different combinations of arbitrary motion of parts of at least one hand;

receiving sensor input sensing motion of parts of a hand of a user in said digital device; and

automatically interpreting said sensor input as a plurality of keystrokes corresponding to keys of a keyboard, said automatically interpreting step being performed by said digital device consistent with a configuration for recognizing hand motion as keys of a keyboard generated by said configuring step.

16. The method for inputting data of claim 15, wherein said sensor input is received from a plurality of sensors mounted in at least one glove fitting said hand of said user.

18. The method for inputting data of claim 15, wherein said different combinations of arbitrary motion of said at least one hand are of significantly less magnitude than hand motion typical for activating keys of a keyboard.

19. The method for inputting data of claim 15, wherein said step of automatically interpreting said sensor input as a plurality of keystrokes comprises processing said sensor input with at least one artificial neural network, said artificial neural network being configured by said configuring step.

20. The method for inputting data of claim 15, wherein said step of automatically interpreting said sensor input as a plurality of keystrokes comprises the steps of:

detecting the occurrence of hand movements corresponding to virtual keystrokes by said user; and

decoding sensor data from said plurality of sensors, said sensor data being coincident with the detection of virtual keystrokes by said detecting step, as said keystrokes corresponding to keys of a keyboard.

21. The method for inputting data of claim 20, wherein said step of detecting the occurrence of hand movements corresponding to virtual keystrokes comprises detecting maxima of a pre-defined deflection measure during respective windows, said pre-defined deflection measure being a function of data from said plurality of sensors.

22. The method for inputting data of claim 21, wherein said window is configured in said configuring step.

23. The method for inputting data of claim 21, wherein said pre-defined deflection measure is of the formula:

17. A method for inputting data to a digital device, comprising the steps of:

configuring said digital device to recognize different combinations of arbitrary motion of parts of at least one hand of an individual user as activations of respective different keys of a keyboard, at least some of said different combinations of arbitrary motion of said at least one hand not touching any other object;

receiving sensor input sensing motion of parts of a hand of a user in said digital device; and

automatically interpreting said sensor input as a plurality of keystrokes corresponding to keys of a keyboard, said automatically interpreting step being performed by said digital device consistent with a configuration for recognizing hand motion as keys of a keyboard generated by said configuring step;

wherein said sensor input is received from a plurality of sensors mounted in at least one glove fitting said hand of said user;

wherein said plurality of sensors comprise a plurality of strain gauges having variable electrical resistances.

24. A program product for managing the input of data to a digital device, comprising:

a plurality of processor executable instructions recorded on signal-bearing media, wherein said instructions, which executed by at least one processor of said digital device, cause the device to perform the steps of;

(a) configuring said digital device to recognize different combinations of arbitrary motion of parts of at least one hand of an individual user as activations of respective different keys of a keyboard, at least some of said different combinations of arbitrary motion of said at least one hand neither touching any other object nor assuming any pre-determined configuration with respect to sensing apparatus for sensing said different combinations of arbitrary motion of parts of at least one hand;

(b) receiving sensor input sensing motion of parts of a hand of a user in said digital device; and

(c) automatically interpreting said sensor input as a plurality of keystrokes corresponding to keys of a keyboard, said automatically interpreting step being performed by said digital device consistent with a configuration for recognizing hand motion as keys of a keyboard generated by said configuring step.

25. The program product of claim 24, wherein said sensor input is received from a plurality of sensors mounted in at least one glove fitting said hand of said user.

26. The program product of claim 25, wherein said plurality of sensors comprise a plurality of strain gauges having variable electrical resistances.

27. The program product of claim 24, wherein said different combinations of arbitrary motion of said at least one hand are of significantly less magnitude than hand motion typical for activating keys of a keyboard.

28. The program product of claim 24, wherein said step of automatically interpreting said sensor input as a plurality of keystrokes comprises processing said sensor input with at least one artificial neural network, said artificial neural network being configured by said configuring step.

29. The program product of claim 24, wherein said step of automatically interpreting said sensor input as a plurality of keystrokes comprises the steps of:

detecting the occurrence of hand movements corresponding to virtual keystrokes by said user; and

decoding sensor data from said plurality of sensors, said sensor data being coincident with the detection of virtual keystrokes by said detecting step, as said keystrokes corresponding to keys of a keyboard.

30. The program product of claim 29, wherein said step of detecting the occurrence of hand movements corresponding to virtual keystrokes comprises detecting maxima of a pre-defined deflection measure during respective windows, said pre-defined deflection measure being a function of data from said plurality of sensors.

31. The program product of claim 30, wherein said window is configured in said configuring step.

32. A program product for managing the input of data to a digital device, comprising:

a plurality of processor executable instructions recorded on signal-bearing media, wherein said instructions, which executed by at least one processor of said digital device, cause the device to perform the steps of;

(a) configuring said digital device to recognize different combinations of arbitrary motion of parts of at least one hand of an individual user as activations of respective different keys of a keyboard, at least some of said different combinations of arbitrary motion of said at least one hand not touching any other object;

(b) receiving sensor input sensing motion of parts of a hand of a user in said digital device; and

(c) automatically interpreting said sensor input as a plurality of keystrokes corresponding to keys of a keyboard, said automatically interpreting step being performed by said digital device consistent with a configuration for recognizing hand motion as keys of a keyboard generated by said configuring step;

wherein said step of automatically interpreting said sensor input as a plurality of keystrokes comprises the steps of;

detecting the occurrence of hand movements corresponding to virtual keystrokes by said user; and

decoding sensor data from said plurality of sensors, said sensor data being coincident with the detection of virtual keystrokes by said detecting step, as said keystrokes corresponding to keys of a keyboard;

wherein said step of detecting the occurrence of hand movements corresponding to virtual keystrokes comprises detecting maxima of a pre-defined deflection measure during respective windows, said pre-defined deflection measure being a function of data from said plurality of sensors;

wherein said pre-defined deflection measure is of the formula;

33. A program product for managing the input of data to a digital device, comprising:

a plurality of processor executable instructions recorded on signal-bearing media, wherein said instructions, which executed by at least one processor of said digital device, cause the device to perform the steps of;

(a) configuring said digital device to recognize different combinations of arbitrary motion of parts of at least one hand of an individual user as activations of respective different keys of a keyboard, at least some of said different combinations of arbitrary motion of said at least one hand not touching any other object;

(b) receiving sensor input sensing motion of parts of a hand of a user in said digital device; and

(c) automatically interpreting said sensor input as a plurality of keystrokes corresponding to keys of a keyboard, said automatically interpreting step being performed by said digital device consistent with a configuration for recognizing hand motion as keys of a keyboard generated by said configuring step;

wherein said program product comprises a first portion for executing on a processor of a portable sensor receiver device attached to said user and a second portion for executing on a general purpose computing device.

34. A method for initializing a data input device, comprising the steps of:

determining a quiescent state for a plurality of sensors of said data input device, said sensors sensing motion of parts of at least one human hand of a user;

prompting said user to form a plurality of different virtual keystrokes by voluntary motion of different respective combinations of parts of said at least one hand, a choice of voluntary motion of different respective combinations of parts of said at least one hand being arbitrary with the user;

sensing the user'"'"'s hand motion responsive to said prompting step with a plurality of sensors;

identifying said plurality of different virtual keystrokes as sensor deviations from said quiescent state for said plurality of sensors; and

configuring said data input device to recognize future sensor input similar to sensor deviations from said quiescent state corresponding to said virtual keystrokes identified by said identifying step as the corresponding respective virtual keystrokes.

40. The method for initializing a data input device of claim 34, wherein said step of configuring said data input device comprises:

(a) configuring a virtual keystroke detector which detects the occurrence of hand movements corresponding to virtual keystrokes by the user; and

(b) configuring a key decoder which decodes information from said plurality of sensors coincident with the detection of virtual keystrokes by said virtual keystroke detector as said keys of a keyboard.

35. The method for initializing a data input device of step 34, wherein said prompting step is performed in a plurality of stages of progressively increasing complexity.

36. The method for initializing a data input device of step 34, wherein said plurality of sensors are mounted in at least one glove fitting said at least one hand.

37. The method for initializing a data input device of step 34, wherein said step of configuring said data input device comprises training at least one artificial neural network using sensor data from at least some of said sensor deviations identified by said identifying step.

38. The method for initializing a data input device of step 34, wherein said step of configuring said data input device comprises calibrating the size of a window for identifying a peak deflection measure as a virtual keystroke.

39. The method for initializing a data input device of step 34, wherein said step of configuring said data input device comprises calibrating the a plurality of coefficients associated with the output of each respective sensor.