Strain-sensing goniometers, systems and recognition algorithms

• US 5,442,729 A
• Filed: 11/08/1993
• Issued: 08/15/1995
• Est. Priority Date: 10/14/1988
• Status: Expired due to Term

• Current Assignee: Board of Trustees of the Leland Stanford Junior University (Stanford Management Co.)
• Original Assignee: Board of Trustees of the Leland Stanford Junior University (Stanford Management Co.)
• Inventors: Kramer, James F., George, William R., Lindener, Peter
• Primary Examiner(s): Knepper, David D.
• Application Number:
  US08/148,991
• Time in Patent Office: 645 Days
• Field of Search: 395/21, 395/2, 395/2.8, 395/22, 340/870.38, 338/210, 128/782
• US Class Current: 704/271
• CPC Class Codes: A61B 2562/0261 Strain gauges
• CPC Class Codes: A61B 5/1071 measuring angles, e.g. using goniometers A61B5/1123 takes precedence
• CPC Class Codes: A61B 5/6806 Gloves
• CPC Class Codes: A61B 5/7267 involving training the classification device
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/014 Hand-worn input/output arrangements, e.g. data gloves
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G09B 21/009 Teaching or communicating with deaf persons G09B21/04 takes precedence; deaf and dumb persons G09B21/00
• CPC Class Codes: G09B 21/04 Devices for conversing with the deaf-blind
• CPC Class Codes: G16H 50/70 for mining of medical data, e.g. analysing previous cases of other patients
• CPC Class Codes: Y10S 128/925 Neural network

Abstract

Goniometers are provided having internal compensation employing two opposing variable resistance strain sensing elements separated by a flexible film. The goniometers may be used in sets for detecting complex hinge or joint movements, where patterns of hinge or joint positions may be used to define symbols. Algorithms are provided for parsing values from a system of goniometers in motion having information content.

Claims

1. A communication system comprising:

means for deriving electrical signals indicative of a configuration of a hand fitted with an instrumented glove having a cover material configured to fit on the hand of a wearer, a plurality of angle-sensing means supported by said cover material and positioned to respond to movement of fingers and/or hand and means for electrically connecting said plurality of angle-sensing means to circuitry of the communication system;

a computer for receiving said electrical signals representing hand-state vectors and determining symbols corresponding to hand-poses, by adaptively signal processing said hand-state vectors to recognize patterns representing said hand-poses; and

first output means responsive to said computer for providing an output of said symbols.

2. The communication system according to claim 1, wherein said angle-sensing means are goniometers which comprise an inert thin electrically non-conductive flexible support backing;

and

first and second variable resistance strain sensing elements affixed on opposite side of said backing; and

means for electrically connecting each of said elements.

3. The communication system according to claim 1, wherein said adaptive signal processing uses a generalized distance metric to determine the distance between a sample hand-state vector and a hand-pose of between two hand-poses, where the generalized distance metric squared is defined by the equation:

space="preserve" listing-type="equation">d.sub.i.sup.2 =r.sub.i.sup.2 +ln|Si|-2*ln, where r.sub.i.sup.2 =(x-m.sub.i).sup.t S.sub.i.sup.-1 ( x-m.sub.i)

and is called the squared Mahalanobis distance for hand-pose i, x is the random hand-state vector, m_i is the vector-valued mean of the samples for hand-pose i, S_i is the sample covariance matrix of the random hand-state vectors for hand-pose i and P(Θ

_i) is the a priori probability of hand-pose i being the intended hand-pose.

4. The communication system according to claim 1, wherein recognition is permitted when the generalized distance from the hand-state vector to the nearest hand-pose is less than the generalized distance from the hand-state vector to the second closest hand-pose by more than a recognition confidence margin.

5. The communication system according to claim 1, wherein said adaptive signal processing further includes a recognition threshold value wherein if the generalized distance from the hand-state vector to a hand-pose is less than said threshold value, said hand-pose may be recognized.

6. The communication system according to claim 5, wherein said adaptive signal processing further includes an unrecognition threshold value greater than said recognition threshold value, wherein if the generalized distance from the hand-state vector to a recognized hand-pose is greater than said unrecognition threshold, said recognized hand-pose is unrecognized.

7. The communication system according to claim 1, wherein said adaptive signal processing unrecognizes the currently recognized hand-pose whenever the generalized distance from the hand-state vector to any hand-pose is less than the generalized distance from the hand-state vector to the currently recognized hand-pose by more than a hand-pose-to-hand-pose hysteresis margin.

8. The communication system according to claim 1, wherein said adaptive signal processing detects pauses, where a pause is indicated when the hand-state vector is currently unpaused and at least one of the following is true:

a) a weighted norm of the hand-state velocity vector falls below a pause-velocity threshold;

b) a weighted norm of the hand-state velocity vector falls a predetermined percentage below the maximum velocity reached since unpaused;

or

c) the deceleration of the hand-state vector averaged over a predetermined time period is greater than a predetermined amount, wherein the average deceleration is defined as the amount of reduction in a weighted norm of the hand-state velocity vector divided by the time over which the hand-state velocity reduction is measured.

9. The communication system according to claim 1, wherein said adaptive signal processing for determining the next nearest hand-pose starts by calculating the generalized distance to the most recent nearest hand-pose, then calculates the generalized distance to the most recent second nearest hand-pose, and then proceeds by checking the remaining hand-poses in order of decreasing probability of occurrence, where each generalized distance calculation is prematurely terminated whenever the running sum total of the weighted squares of individual joint errors exceeds the calculated generalized distance squared from the hand-state to the current second closest hand-pose.

10. A communication system according to claim 1, wherein the said adaptive signal processing incorporates a feedforward artificial neural network classifier.

11. A communication system comprising:

means for deriving electrical signals indicative of a configuration of a hand fitted with an instrumented glove having a cover material configured to fit on the hand of a wearer, a plurality of goniometers supported by said cover material and positioned to flex with movement of fingers and/or hand and means for electrically connecting said plurality of goniometers to circuitry of the communication system;

a computer for receiving said electrical signals representing said hand-state vectors and determining symbols corresponding to hand-poses, by adaptively signal processing said hand-state vectors to recognize patterns representing hand-poses; and

first output means responsive to said computer for providing an output of said symbols;

wherein said adaptive signal processing uses one of the following classification algorithms with the associated distance measure;

A) a generalized distance measure to determine the distance between a sample hand-state vector and a hand-pose of between two hand-poses, where the generalized distance measure squared is defined by the equation;

space="preserve" listing-type="equation">d.sub.i.sup.2 =r.sub.i.sup.2 +ln|S.sub.i |-2*ln;

space="preserve" listing-type="equation">where r.sub.i.sup.2 =(x-m.sub.i).sup.t S.sub.i.sup.-1 (x-m.sub.i)

and is called the squared Mahalanobis distance for hand-pose i, x is the random hand-state vector, m_i is the vector-valued mean of the samples for hand-pose i, S_i is the sample covariance matrix of the random hand-state vectors for hand-pose i and P(Θ

_i) is the a priori probability of hand-pose i being the intended hand-pose;

or

B) a feedforward artificial neural network classifier, which utilizes a function of the value output from the sigmoid function of the unit of the output layer which corresponds to a particular class as the distance measure between the random hand-state vector in question and the representative hand-pose of said class;

wherein said adaptive signal processing further includes a recognition threshold value, wherein if the distance measure from the hand-state vector to a hand-pose is less than said threshold value, said hand-pose may be recognized; and

wherein said adaptive signal processing further includes at least one of the following;

a) with the proviso that the distance measure of parts A or B is used, recognition is permitted when the distance measure from the hand-state vector to the nearest hand-pose is less than the distance measure from the hand-state vector to the second closest hand-pose by more than a recognition confidence margin;

b) an unrecognition threshold value greater than said recognition threshold value, wherein if the distance measure from the hand-state vector to a recognized hand-pose is greater than said unrecognition threshold, said recognized hand-pose is unrecognized;

c) the currently recognized hand-pose is unrecognized whenever the distance measure from the hand-state vector to any hand-pose is less than the distance measure from the hand-state vector to the currently recognized hand-pose by more than a hand-pose-to-hand-pose hysteresis margin;

d) a pause detection means, where a pause is indicated when the hand-state vector is currently unpaused and at least one of the following is true;

i) a weighted norm of the hand-state velocity vector falls below a pause-velocity threshold;

ii) a weighted norm of the hand-state velocity vector falls a predetermined percentage below the maximum velocity reached since unpaused;

or

iii) the deceleration of the hand-state vector averaged over a predetermined time period is greater than a predetermined amount, wherein the average deceleration is defined as the amount of reduction in a weighted norm of the hand-state velocity vector divided by the time over which the hand-state velocity reduction is measured;

e) with the proviso that the generalized distance measure of part A is used, each generalized distance calculation is prematurely terminated whenever the running sum total of the weighted squares of individual joint errors exceeds a critical distance.

12. A communication system according to claim 11, wherein in relation to part e, the search order calculation for determining the next nearest hand-pose proceeds by checking the hand-poses in order of decreasing probability of occurrence.

13. A communication system according to claim 11, wherein the critical distance of part e is the distance from the random hand-state vector to the most recent closest or second closest hand-pose;

or some predefined maximum acceptable distance.

14. A method for recognizing hand poses employing an adaptive recognition algorithm, said method comprising:

producing electrical signals representing hand-state vectors indicative of a configuration of a hand fitted with an instrumented glove comprising a cover material configured to fit on the hand of a wearer, and a plurality of angle-sensing means supported by said cover material and positioned to respond to movement of fingers and/or hand;

determining symbols from said electrical signals representing hand-state vectors corresponding to hand-poses, by adaptively signal processing said hand-state vectors to recognize patterns representing said hand-poses; and

outputing said symbols representing said hand-poses.

15. A method for recognizing hand-poses employing an adaptive recognition algorithm, said method comprising:

producing electrical signals representing hand-state vectors indicative of a configuration of a hand fitted with an instrumented glove comprising a cover material configured to fit on the hand of a wearer, and a plurality of angle-sensing means supported by said cover material and positioned to respond to movement of fingers and/or hand;

determining symbols corresponding to hand-poses from said electrical signals, by adaptively signal processing said hand-state vectors to recognize patterns representing hand-poses; and

outputing said symbols representing said hand-poses;

wherein said adaptive signal processing uses one of the following classification algorithms with the associated distance measure;

A) a generalized distance measure to determine the distance between a sample hand-state vector and a hand-pose of between two hand-poses, where the generalized distance measure squared is defined by the equation;

space="preserve" listing-type="equation">d.sub.i.sup.2 =r.sub.i.sup.2 +ln|S.sub.i |-2*ln;

space="preserve" listing-type="equation">where r.sub.i.sup.2 =(x-m.sub.i).sup.t S.sub.i.sup.-1 (x-m.sub.i)

and is called the squared Mahalanobis distance for hand-pose i, x is the random hand-state vector, m_i is the vector-valued mean of the samples for hand-pose i, S_i is the sample covariance matrix of the random hand-state vectors for hand-pose i and P(Θ

_i) is the a priori probability of hand-pose i being the intended hand-pose;

or

B) a feedforward artificial neural network classifier, which utilizes a function of the value output from the sigmoid function of the unit of the output layer which corresponds to a particular class as the distance measure between the random hand-state vector in question and the representative hand-pose of said class;

wherein said adaptive signal processing further includes a recognition threshold value, wherein if the distance measure from the hand-state vector to a hand-pose is less than said threshold value, said hand-pose may be recognized; and

wherein said adaptive signal processing further includes at least one of the following;

a) with the proviso that the distance measure of parts A or B is used, recognition is permitted when the distance measure from the hand-state vector to the nearest hand-pose is less than the distance measure from the hand-state vector to the second closest hand-pose by more than a recognition confidence margin;

b) an unrecognition threshold value greater than said recognition threshold value, wherein if the distance measure from the hand-state vector to a recognized hand-pose is greater than said unrecognition threshold, said recognized hand-pose is unrecognized;

c) the currently recognized hand-pose is unrecognized whenever the distance measure from the hand-state vector to any hand-pose is less than the distance measure from the hand-state vector to the currently recognized hand-pose by more than a hand-pose-to-hand-pose hysteresis margin;

d) a pause detection means, where a pause is indicated when the hand-state vector is currently unpaused and at least one of the following is true;

i) a weighted norm of the hand-state velocity vector falls below a pause-velocity threshold;

ii) a weighted norm of the hand-state velocity vector falls a predetermined percentage below the maximum velocity reached since unpaused;

or

iii) the deceleration of the hand-state vector averaged over a predetermined time period is greater than a predetermined amount, wherein the average deceleration is defined as the amount of reduction in a weighted norm of the hand-state velocity vector divided by the time over which the hand-state velocity reduction is measured;

e) with the proviso that the generalized distance measure of part A is used, each generalized distance calculation is prematurely terminated whenever the running sum total of the weighted squares of individual joint errors exceeds a critical distance.