System and method for recognizing touch typing under limited tactile feedback conditions

• US 6,677,932 B1
• Filed: 01/28/2001
• Issued: 01/13/2004
• Est. Priority Date: 01/28/2001
• Status: Expired

• Current Assignee: Apple Inc.
• Original Assignee: Finger Works Incorporated
• Inventors: Westerman, Wayne Carl
• Primary Examiner(s): Shalwala, Bipin
• Assistant Examiner(s): Osorio, Ricardo
• Application Number:
  US09/681,146
• Time in Patent Office: 1,080 Days
• Field of Search: 345/156, 345/157, 345/168, 345/171-173, 382/181, 382/187, 382/189, 706/12, 706/15, 706/17, 706/18, 706/20, 706/26
• US Class Current: 345/173
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus

Abstract

A system is disclosed for recognizing typing from typing transducers that provide the typist with only limited tactile feedback of key position. The system includes a typing decoder sensitive to the geometric pattern of a keystroke sequence as well as the distance between individual finger touches and nearby keys. The typing decoder hypothesizes plausible key sequences and compares their geometric pattern to the geometric pattern of corresponding finger touches. It may also hypothesize home row key locations for touches caused by hands resting on or near home row. The resulting pattern match metrics may be combined with character sequence transition probabilities from a spelling model. The typing decoder then chooses the hypothesis sequence with the best cumulative match metric and sends it as key codes or commands to a host computing device.

Claims

1. A typing recognition apparatus for touch typing on surfaces with limited tactile feedback that compensates for finger and hand drift during typing and discourages any integrated spelling model from choosing dictionary words over unusual but carefully typed strings, the apparatus comprising:

a typing surface means that displays symbols indicating the locations of touchable keys;

touch sensor means that provides the surface coordinates of each touch by a typist attempting to strike said key symbols on said surface;

hypothesis tree generator means that extends existing key hypothesis sequences with hypotheses for keys in the neighborhood of each new touch;

pattern geometry evaluation means that computes geometry match metrics for the hypothesized key sequences by comparing separation vectors between the successive touch locations with separation vectors between the successively hypothesized key locations as well as by measuring the zero-order key/touch alignment error;

decoding means that finds the hypothesized key sequence with the best cumulative match metric; and

,

transmission means for communicating the symbols and commands represented by the best hypothesized key sequence to host computer applications.

2. The apparatus of claim 1 wherein a synchronization detection means inserts resting finger hypothesis into the hypothesis tree upon detection of a hand resting substantially on home row, and wherein said resting hypotheses are given for key separation vector computation purposes the coordinates of the home row key that their touch'"'"'s identified finger normally rests upon.

3. The apparatus of claim 1 wherein a stack decoder is utilized as the particular decoding means.

4. The apparatus of claim 1 wherein the geometry match metric for a hypothesized key is substantially formulated as the squared distance between a touch and its hypothesized key plus the sum of squared differences between corresponding key and touch separation vectors of all valid orders.

5. The apparatus of claim 4 wherein the difference between a touch separation vector and the corresponding key separation vector is weighted in roughly inverse proportion to the touch time difference between the two touches from which the touch separation vector was computed.

6. The apparatus of claim 4 wherein the difference between a touch separation vector and the corresponding key separation vector is weighted less if the touch separation vector is large.

7. A method for recognizing typing from typing devices that sense lateral finger position but provide limited tactile feedback of key location, the method advantageously compensating for finger and hand drift during typing and discouraging any integrated spelling model from choosing dictionary words over unusual but carefully typed strings, wherein the method comprises the following steps:

forming a touch location and time sequence from the fingertip position at the end of each keystroke as measured by typing sensors;

computing a set of touch separation vectors of increasing orders from the location difference between the newest touch and previous touch in said touch location sequence;

generating a set of key hypothesis sequences for the given touch sequence, each hypothesis in a sequence being for a key near the location of the touch causing the hypothesis;

for each key hypothesis, computing a set of key separation vectors of increasing orders from differences between the position of the newest key and previous keys in the hypothesized sequence;

for each key hypothesis, computing a geometry match metric as a function of the magnitude of the zero-order touch/key alignment error as well as of the magnitudes of each order'"'"'s touch and key separation vector difference;

combining the geometry match metrics from each hypothesis in a key hypothesis sequence into a cumulative match metric for the hypothesis sequence;

choosing the hypothesized key sequence with the best cumulative metric as the best hypothesized key sequence; and

,

transmitting the symbols and commands represented by the best hypothesized key sequence to a host computer for further action.

8. The method of claim 7 wherein the magnitude of each difference between a touch separation vector and the corresponding key separation vector is weighted in roughly inverse proportion to the time between the two touches from which the touch separation vector was computed.

9. The method of claim 7 wherein the magnitude of each difference between a touch separation vector and the corresponding key separation vector is weighted less if the touch separation vector is large.

10. The method of claim 7 wherein a synchronization detection means inserts resting finger hypothesis into the hypothesis tree upon detection of a hand resting substantially on home row, and wherein said resting hypotheses are given for key separation vector computation purposes the coordinates of the home row key that their touch'"'"'s identified finger normally rests upon.

11. The method of claim 7 wherein the set of key hypothesis sequences are stored as a hypothesis tree that can extend the sequences upon reception of a new touch by sprouting new hypotheses.

12. The method of claim 11 wherein a stack decoder is utilized to find the best hypothesized key sequence.

13. A method for recognizing typing from typing devices that sense lateral finger position but provide limited tactile feedback of key location, the method advantageously compensating for finger and hand drift during typing and discouraging any integrated spelling model from choosing dictionary words over unusual but carefully typed strings, wherein the method comprises the following steps:

forming a touch location and time sequence from the fingertip position at the end of each keystroke as measured by typing sensors;

generating a set of key hypothesis sequences for the given touch sequence, each hypothesis in a sequence being for a key near the location of the touch causing the hypothesis;

for each key hypothesis, computing a key/touch alignment error vector as the difference between the location of the hypothesized key and the location of its causing touch;

for each key hypothesis, computing a geometry match metric as a function of the magnitude of the hypothesis'"'"' key/touch alignment error as well as of the magnitude of differences between the hypothesis'"'"' key/touch alignment error vector and that of preceding hypotheses in its sequence;

combining the geometry match metrics from each hypothesis in a key hypothesis sequence into a cumulative match metric for the hypothesis sequence;

choosing the hypothesized key sequence with the best cumulative metric as the best hypothesized key sequence; and

,

transmitting the symbols and commands represented by the best hypothesized key sequence to a host computer for further action.

14. The method of claim 13 wherein the magnitude of the difference between two hypotheses'"'"' key/touch alignment error vectors is weighted in roughly inverse proportion to the time between the two touches from which the touch separation vector was computed.

15. The method of claim 13 wherein the magnitude of the difference between two hypotheses'"'"' key/touch alignment error vectors is weighted less if the separation between the corresponding touches is large.

16. The method of claim 13 wherein a synchronization detection means inserts resting finger hypothesis into the hypothesis tree upon detection of a hand resting substantially on home row, and wherein said resting hypotheses are given for key/touch alignment error vector computation purposes the coordinates of the home row key that their touch'"'"'s identified finger normally rests upon.

17. The method of claim 13 wherein the set of key hypothesis sequences are stored as a hypothesis tree that can extend the sequences upon reception of a new touch by sprouting new hypotheses.

18. The method of claim 17 wherein a stack decoder is utilized to find the best hypothesized key sequence.