Interactive animation system for sign language

• US 20060134585A1
• Filed: 08/31/2005
• Published: 06/22/2006
• Est. Priority Date: 09/01/2004
• Status: Abandoned Application

• Current Assignee: Gerardo Beni, Marie A. Nadolske, Nicoletta Adamo-Villani, Ronnie Wilbur
• Original Assignee: Gerardo Beni, Marie A. Nadolske, Nicoletta Adamo-Villani, Ronnie Wilbur
• Inventors: Nadolske, Marie A., Wilbur, Ronnie, Beni, Gerardo, Adamo-Villani, Nicoletta
• Application Number:
  US11/216,606
• Publication Number:
  US 20060134585A1
• Time in Patent Office: Days
• US Class Current: 434/112
• CPC Class Codes: G09B 21/009 Teaching or communicating with deaf persons G09B21/04 takes precedence; deaf and dumb persons G09B21/00
• CPC Class Codes: G09B 5/02 with visual presentation of the material to be studied, e.g. using film strip

Abstract

A method and system for interactive communication in sign language using computer animation. In one aspect, a user interface is provided with a first activity area and a second activity area. A three-dimensional avatar configured to communicate using sign language is displayed between the first activity area and the second activity area. In response to the user selection of a respective one of the activity areas, the avatar is directed to sign an expression associated with the selected activity area. In another aspect, a method of teaching mathematics using sign language is provided. According to another aspect, a method of animating a signed communication is provided. In another aspect, a method of creating an animation of a sign language expression is provided.

Claims

1. In a computer system having a graphical user interface including a display and a selection device, a method comprising:

retrieving a first set of elements for a first activity area;

retrieving a second set of elements for a second activity area;

displaying the first activity area;

displaying the second activity area;

displaying a three-dimensional avatar configured to communicate using sign language on the display between the first activity area and the second activity area;

receiving a selection signal indicative of the selection device pointing at a respective element of the first set of elements;

in response to the selection signal, directing the avatar to sign an expression associated with the selected element of the first set of elements.

2. The method of claim 1, where the first activity area and the avatar are spaced apart such that a user can visually focus simultaneously on at least a portion of the first activity area and the avatar.

3. The method of claim 1, where the second activity area and the avatar are spaced apart such that a user can visually focus simultaneously on at least a portion of the second activity area and the avatar.

4. The method of claim 1, where the first activity area and the avatar are spaced apart such that a user can visually focus on at least a portion of the first activity area and interpret a signed communication from the avatar without any eye movement.

5. The method of claim 1, further comprising displaying an explanation of a function associated with at least one element of the first set of elements using sign language.

6. The method of claim 1, where the avatar is configured to communicate a sign language expression by retrieving a sign language animation segment from a library containing a plurality of sign language animation segments.

7. The method of claim 6, where at least a portion of the sign language animation segments stored in the library were captured from a motion capture glove.

8. The method of claim 7, where at least a portion of the sign language animation segments stored in the library were captured from a motion capture suit.

9. In a computer system having a graphical user interface including a display and a selection device, a method of teaching mathematics using sign language comprising:

displaying a three-dimensional avatar configured to communicate using sign language on the display;

retrieving a mathematical problem;

displaying the mathematical problem on the display in a textual manner; and

directing the avatar to communicate the mathematical problem using sign language.

10. The method of claim 9, further comprising receiving a selection signal indicative of the selection device pointing at the mathematical problem and in response to the signal, directing the avatar to communicate the mathematical problem using sign language.

11. The method of claim 9, further comprising displaying more than one possible answer to the mathematical problem and directing the avatar to communicate the possible answers using sign language.

12. The method of claim 9, where in response to receiving a proposed answer to the mathematical problem from a user, determining whether the proposed answer is correct and directing the avatar to indicate whether the proposed answer is correct using sign language.

13. The method of claim 12, further comprising retrieving an explanation associated with the mathematical problem and directing the avatar to communicate the explanation using sign language.

14. The method of claim 9, where the mathematical problem is selected from the group consisting of addition, subtraction, division and multiplication.

15. A method of animating a signed communication:

providing a first animation segment configured to sign a first expression, where a signer in the first animation segment starts in a first position;

providing a second animation segment configured to sign the first expression, where a signer in the second animation segment starts in a second position;

receiving a request to sign the first expression;

determining whether the first expression will be an initial segment in an animation sequence,

if the first expression is the initial segment in the animation sequence, retrieving the first animation segment, and

if the first expression is not the initial segment in the animation sequence, retrieving the second animation segment.

16. The method of claim 15, where the first position is a neutral pose.

17. The method of claim 15, where the signer has a chest and where in the second position the signer has at least one hand in front of the chest.

18. The method of claim 15, further comprising:

providing a third animation segment configured to sign the first expression, where a signer in the third animation segment starts in a third position;

providing a fourth animation segment configured to sign the first expression, where a signer in the fourth animation segment starts in a fourth position;

determining whether the first expression will be a final segment in an animation sequence;

retrieving the third animation segment if the first expression is a final segment in an animation sequence.

19. The method of claim 18, further comprising retrieving the fourth animation segment if the first expression is not the final segment in the animation sequence.

20. A method of animating a signed communication, the method comprising:

providing a library of sign language expressions, each expression associated a first animation segment, a second animation segment, a third animation segment and a fourth animation segment, where a signer in the first animation segment starts in a first position and ends in the first position, where a signer in the second animation segment starts in a second position and ends in the second position, where a signer in the third animation segment starts in the first position and ends in the second position and a signer in the fourth animation segment starts in the second position and ends in the first position;

receiving a request for a signed communication, the signed communication including an initial expression, an intermediate expression and a final expression;

retrieving the first animation segment for a sign language expression corresponding to the initial expression from the library;

displaying the first animation segment on a display;

retrieving the fourth animation segment for a sign language expression corresponding to the intermediate expression from the library;

displaying the fourth animation segment on the display;

retrieving the second animation segment for a sign language expression corresponding to the final expression from the library; and

displaying the second animation segment on the display.

21. A method of creating an animation of a sign language expression, the method comprising:

capturing a first signal from a motion capture suit, the first signal representing a range of motion during a signed expression;

capturing a second signal from a motion capture glove, the second signal representing a range of motion during the signed expression;

converting the first signal and the second signal into an animation sequence in which an avatar communicates the signed expression.