METHOD AND APPARATUS FOR RECOGNIZING SIGN LANGUAGE USING ELECTROMYOGRAM SENSOR AND GYRO SENSOR

• US 20130295529A1
• Filed: 10/28/2011
• Published: 11/07/2013
• Est. Priority Date: 12/10/2010
• Status: Active Grant

• Current Assignee: Foundation of Soongsil University-Industry Cooperation
• Original Assignee: Foundation of Soongsil University-Industry Cooperation
• Inventors: Shin, Hyun Chool, You, Kyung Jin, Rhee, Ki Won, Kang, Hee Su
• Granted Patent: US 9,183,760 B2
• Time in Patent Office: Days
• US Class Current: 434/112
• CPC Class Codes: A61B 2562/0219 Inertial sensors, e.g. accelerometers, gyroscopes, tilt switches
• CPC Class Codes: A61B 5/1114 Tracking parts of the body
• CPC Class Codes: A61B 5/1125 Grasping motions of hands
• CPC Class Codes: A61B 5/389 Electromyography [EMG]
• CPC Class Codes: A61B 5/6825 Hand
• CPC Class Codes: G06F 2218/12 Classification; Matching
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113
• CPC Class Codes: G09B 21/009 Teaching or communicating with deaf persons G09B21/04 takes precedence; deaf and dumb persons G09B21/00

Abstract

Provided are a method and apparatus for recognizing sign language using an electromyogram sensor and a gyro sensor. The method includes the steps of (a) receiving a gyro measurement signal and an electromyogram measurement signal from the gyro sensor and the electromyogram sensor which are attached to the body of a subject, (b) determining to which of groups clustered with similar fingerspelling motions the gyro measurement signal belongs, (c) acquiring a Gaussian model for the electromyogram measurement signal, and (d) comparing the acquired Gaussian model with Gaussian candidate models for candidate fingerspelling motions which belong to the group determined in step (b), and recognizing the candidate fingerspelling motion corresponding to the Gaussian candidate model that is most similar to the acquired Gaussian model as the fingerspelling motion of the subject.

Claims

1. A method for recognizing sign language using an electromyogram sensor and a gyro sensor, the method comprising the steps of:

(a) receiving a gyro measurement signal and an electromyogram measurement signal from the gyro sensor and the electromyogram sensor which are attached to the body of a subject;

(b) determining to which of groups clustered with similar fingerspelling motions the gyro measurement signal belongs;

(c) acquiring a Gaussian model for the electromyogram measurement signal; and

(d) comparing the acquired Gaussian model with Gaussian candidate models for candidate fingerspelling motions which belong to the group determined in step (b), and recognizing the candidate fingerspelling motion corresponding to the Gaussian candidate model that is most similar to the acquired Gaussian model as the fingerspelling motion of the subject.

2. The method according to claim 1, wherein step (b) further includes clustering the fingerspelling motions, which are similar in characteristic of the signal measured by the gyro sensor, into groups, the group-specific clustering includes:

(b1) obtaining rotational angle coordinate samples of each fingerspelling motion using roll and pitch rotation values acquired from the gyro sensor;

(b2) measuring a distance from central coordinates set for each group with respect to each of the acquired rotational angle coordinate samples and allocating the rotational angle coordinates to a group located at a distance closest to the rotational angle coordinate sample;

(b3) calculating an average value of the allocated rotational angle coordinates and the central coordinates to obtain new central coordinates; and

(b4) repeating steps (b2) and (b3) so that the rotational angle coordinates of the rotational angle coordinate samples are congregated according to each group.

3. The method according to claim 1, wherein step (c) includes:

(c1) obtaining an entropy for each electromyogram measurement signal acquired from at least one of the electromyogram sensors having different channels, and

(c2) obtaining a Gaussian model according to the entropy.

4. The method according to claim 3, wherein step (d) includes calculating individual similarities between the Gaussian model of each electromyogram measurement signal acquired from at least one of the electromyogram sensors and Gaussian candidate models of the fingerspelling motions belonging to the group determined in step (b), and recognizing the candidate fingerspelling motion, in which a product of the calculated individual similarities shows the greatest value, as the fingerspelling motion of the subject.

5. The method according to claim 4, wherein the individual similarities are calculated using a maximum likelihood estimation method.

6. An apparatus for recognizing sign language using an electromyogram sensor and a gyro sensor, the apparatus comprising:

a signal receiving part receiving a gyro measurement signal and an electromyogram measurement signal from the gyro sensor and the electromyogram sensor which are attached to the body of a subject;

a group determining part determining to which of groups clustered with similar fingerspelling motions the gyro measurement signal belongs;

a model acquiring part acquiring a Gaussian model for the electromyogram measurement signal; and

sign language recognizing part comparing the acquired Gaussian model with Gaussian candidate models for candidate fingerspelling motions which belong to the group determined by the group determining part, and recognizing the candidate fingerspelling motion corresponding to the Gaussian candidate model that is most similar to the acquired Gaussian model as the fingerspelling motion of the subject.

7. The apparatus according to claim 6, further comprising a clustering part clustering the fingerspelling motions, which are similar in characteristic of the signal measured by the gyro sensor, into groups,

wherein the clustering part obtains rotational angle coordinate samples of each fingerspelling motion using roll and pitch rotation values acquired from the gyro sensor, measures a distance from central coordinates set for each group with respect to each of the acquired rotational angle coordinate samples of the fingerspelling motions to allocate the rotational angle coordinates to a group located at a distance closest to the rotational angle coordinate sample, calculates an average value of the allocated rotational angle coordinates and the central coordinates to obtain new central coordinates, and repeats the processes of allocating the rotational angle coordinates and acquiring the new central coordinates for each rotational angle coordinate sample so that the rotational angle coordinates of the rotational angle coordinate samples are congregated according to each group.

8. The apparatus according to claim 6, wherein the model acquiring part obtains an entropy for each electromyogram measurement signal acquired from at least one of the electromyogram sensors having different channels, and acquires the Gaussian model according to the entropy.

9. The apparatus according to claim 8, wherein the manual alphabet recognizing part calculates individual similarities between the Gaussian model of each electromyogram measurement signal acquired from at least one of the electromyogram sensors and Gaussian candidate models of the fingerspelling motions belonging to the group determined by the group determining part, and recognizes the candidate fingerspelling motion, in which a product of the calculated individual similarities shows the greatest value, as the fingerspelling motion of the subject.

10. The apparatus according to claim 9, wherein the individual similarities are calculated using a maximum likelihood estimation method.