System for human trajectory learning in virtual environments

• US 5,846,086 A
• Filed: 02/13/1996
• Issued: 12/08/1998
• Est. Priority Date: 07/01/1994
• Status: Expired due to Fees

• Current Assignee: Massachusetts Institute of Technology
• Original Assignee: Massachusetts Institute of Technology
• Inventors: Bizzi, Emilio, Shadmehr, Reza, Mussa-Ivaldi, Ferdinando A.
• Primary Examiner(s): Cheng, Joe H.
• Application Number:
  US08/600,445
• Time in Patent Office: 1,029 Days
• Field of Search: 434/118, 434/247, 434/248, 434/251, 434/252, 434/307 R, 434/308, 434/317, 434/365, 463/1, 463/23, 473/43, 473/446, 473/467, 473/453, 473/461, 482/902, 345/302, 345/473, 345/112, 345/113, 345/156, 364/410, 348/61, 348/578
• US Class Current: 434/247
• CPC Class Codes: A63B 2024/0012 Comparing movements or motion sequences with a registered reference
• CPC Class Codes: A63B 2024/0015 Comparing movements or motion sequences with computerised simulations of movements or motion sequences, e.g. for generating an ideal template as reference to be achieved by the user
• CPC Class Codes: A63B 2071/0647 Visualisation of executed movements
• CPC Class Codes: A63B 2102/16 Table tennis
• CPC Class Codes: A63B 2208/12 specially adapted for children
• CPC Class Codes: A63B 2220/89 Field sensors, e.g. radar systems
• CPC Class Codes: A63B 2243/0033 Handball
• CPC Class Codes: A63B 24/0006 Computerised comparison for qualitative assessment of motion sequences or the course of a movement
• CPC Class Codes: A63B 69/00 Training appliances or apparatus for special sports training of parachutists B64D23/00
• CPC Class Codes: A63B 69/0002 for baseball
• CPC Class Codes: A63B 69/0015 for cricket
• CPC Class Codes: A63B 69/002 for football American-football dummies A63B69/34
• CPC Class Codes: A63B 69/0026 for ice-hockey
• CPC Class Codes: A63B 69/3623 for driving A63B69/3608, A63B69/3658, A63B69/3661, A63B69/3667 and A63B69/3691 take precedence
• CPC Class Codes: A63B 71/0622 Visual, audio or audio-visual systems for entertaining, instructing or motivating the user
• CPC Class Codes: G09B 19/0038 Sports
• CPC Class Codes: Y10S 482/902 Employing specific graphic or video display

Abstract

In an apparatus and method for teaching motor skills to a human subject, learning by a student is based upon imitation of a teacher'"'"'s motion. The teacher'"'"'s motion is sensed by a motion sensing device, is digitized by a computer, and is then recorded in memory. The computer generates a virtual image sequence of the teacher on the computer display of the teacher interacting in the computer-generated virtual environment for the student to view and imitate. The same motion sensing device used to record the teacher'"'"'s movement tracks the student'"'"'s movement and the computer creates a virtual image sequence on the display corresponding to the student'"'"'s movement and its interaction with the virtual environment in real-time, simultaneously providing an image sequence of the teacher'"'"'s movement. The student'"'"'s task is to imitate the teacher'"'"'s movement. This process is repeated until the teacher'"'"'s and student'"'"'s trajectories match well in space and time.

Claims

1. A teaching system comprising:

a motion sensing device for sensing a student'"'"'s movement;

a display;

memory; and

electronics responsive to the motion sensing device to generate, during the student'"'"'s movement, a student'"'"'s virtual image sequence on the display corresponding to the student'"'"'s movement and to simultaneously cause a teacher'"'"'s virtual image sequence on the display of a corresponding teacher'"'"'s movement from memory such that the student may cause the student'"'"'s virtual image sequence to imitate the teacher'"'"'s virtual image sequence.

2. A teaching system as claimed in claim 1, wherein the motion sensing device senses the position and orientation of an implement set in motion by the student.

3. A teaching system as claimed in claim 2, wherein the implement is a racquet.

4. A teaching system as claimed in claim 2, wherein the electronics causes an image sequence of a moving projectile struck by the implement.

5. A system as claimed in claim 4, wherein the projectile of the image sequence has spin, and translational motion.

6. A teaching system as claimed in claim 2 wherein the electronics further displays a prompt to the movement, the prompt comprising an object to be impacted by the implement.

7. A teaching system as claimed in claim 2 wherein the student'"'"'s and teacher'"'"'s virtual image sequences are endpoint trajectories of the implement set in motion without images of the student and teacher.

8. A system as claimed in claim 1, wherein the teacher'"'"'s movement is a prestored acceptable movement of the student.

9. A system as claimed in claim 1, wherein the electronics cause the virtual image sequence of the teacher'"'"'s movement to be displayed at a reduced speed.

10. A system as claimed in claim 9, wherein the electronics increase the speed of the virtual image sequence of the teacher'"'"'movement as the student becomes proficient in following the teacher'"'"'s movement.

11. A system as claimed in claim 1, wherein the teacher'"'"'s movement is selected from a library of stored teacher'"'"'s movements, according to the student'"'"'s needs.

12. A system as claimed in claim 1, wherein the electronics increase the speed of the virtual image sequence of the teacher'"'"'s movement as the student becomes proficient in following the teacher'"'"'s movement.

13. A teaching system as claimed in claim 1 wherein the motion sensing device senses the position and orientation of a member set in motion by the student and wherein the student'"'"'s virtual image sequence is derived from the sensed position and orientation of the member.

14. A teaching system as claimed in claim 1 wherein the student'"'"'s and teacher'"'"'s virtual image sequences are displayed in three-dimensional form to provide a perception of depth.

15. A method for teaching motion, comprising:

sensing a student'"'"'s movement with a motion sensing device;

displaying, during the student'"'"'s movement, a student'"'"'s virtual image sequence in response to the student'"'"'s movement; and

displaying a teacher'"'"'s virtual image sequence of a teacher'"'"'s movement from memory simultaneously with the student'"'"'s virtual image sequence, such that the student may cause the student'"'"'s virtual image sequence to imitate the teacher'"'"'s virtual image sequence.

16. The method of claim 15 wherein the step of sensing a student'"'"'s movement further comprises sensing the position and orientation of an implement set in motion by the student'"'"'s movement.

17. The method of claim 16 wherein the implement is a racquet.

18. The method of claim 16 further comprising the steps of striking a projectile with the implement set in motion by the student and displaying the resulting trajectory of the projectile.

19. The method of claim 18 wherein the step of displaying the trajectory of the projectile takes into account the projectile'"'"'s spin.

20. The method of claim 16 wherein the step of displaying further comprises displaying a prompt to the movement, the prompt comprising an object to be impacted by the implement.

21. The method of claim 16 wherein the step of displaying the student'"'"'s and teacher'"'"'s virtual image sequences further comprises displaying endpoint trajectories of the implement set in motion without images of the student and teacher.

22. The method of claim 15 wherein the step of displaying an image sequence of the teacher'"'"'s movement from memory includes prestoring an acceptable movement of the student as the teacher'"'"'s movement.

23. The method of claim 15, wherein the step of displaying the image sequence of the teacher'"'"'s movement from memory incudes displaying the teacher'"'"'s movement at a reduced speed.

24. The method of claim 23, further comprising the step of increasing the speed of the teacher'"'"'s movement as the student becomes proficient at following the teacher'"'"'s movement.

25. The method of claim 15 wherein the step of displaying an image sequence of a teacher'"'"'s movement from memory includes the step of selecting a teacher'"'"'s movement from a library of stored teacher'"'"'s movements according to the student'"'"'s needs.

26. The method of claim 15 wherein the step of sensing a student'"'"'s movement with a motion sensing device further comprises the step of establishing an oriented electromagnetic field between fixed and moving units of the motion sensing device.

27. The method of claim 15 wherein the step of sensing further comprises sensing the position and orientation of a member set in motion by the student, and further comprising the step of deriving the student'"'"'s virtual image sequence from the sensed position and orientation of the member.

28. The method of claim 15 wherein the steps of displaying the student'"'"'s and teacher'"'"'s virtual images further comprise displaying the images in three-dimensional form to provide a perception of depth.

29. A teaching system comprising:

a motion sensing device for sensing student'"'"'s movement;

a display; and

electronics responsive to the motion sensing device to generate, during the student'"'"'s movement, a student'"'"'s virtual image sequence on the display corresponding to a student'"'"'s movement, and to simultaneously cause a teacher'"'"'s virtual image sequence on the display corresponding to a teacher'"'"'s movement such that the student may cause the student'"'"'s virtual image sequence to imitate the teacher'"'"'s virtual image sequence.

30. A teaching system comprising:

a motion sensing device for sensing a student'"'"'s movement;

a display;

memory; and

electronics responsive to the motion sensing device to generate a student'"'"'s virtual image sequence on the display in real-time corresponding to the student'"'"'s movement and to simultaneously cause a teacher'"'"'s virtual image sequence on the display of a teacher'"'"'s movement from memory such that the student may cause the student'"'"'s virtual image sequence to follow the teacher'"'"'s virtual image sequence.

31. A teaching system as claimed in claim 30, wherein the motion sensing device senses the position and orientation of an implement set in motion by the student.

32. A teaching system as claimed in claim 31, wherein the implement is a racquet.

33. A teaching system as claimed in claim 32, wherein the electronics causes an image sequence of a moving projectile struck by the implement.

34. A system as claimed in claim 33, wherein the projectile of the image sequence has spin, and translational motion.

35. A system as claimed in claim 33 wherein the electronics is adapted to generate an enlarged view of the implement striking the projectile.

36. A system as claimed in claim 31 wherein the electronics is adapted to generate a shadow of the implement.

37. A system as claimed in claim 30, wherein the teacher'"'"'s movement is a prestored acceptable movement of the student.

38. A system as claimed in claim 30, wherein the electronics causes the image sequence of the teacher'"'"'s movement to be displayed at a reduced speed.

39. A system as claimed in claim 38, wherein the electronics increases the speed of the image sequence of the teacher'"'"'s movement as the student becomes proficient in following the teacher'"'"'s movement.

40. A system as claimed in claim 30, wherein the teacher'"'"'s movement is selected from a library of stored teacher'"'"'s movements, according to the student'"'"'s needs.

41. A system as claimed in claim 30, wherein the motion sensing device establishes an oriented electromagnetic field between fixed and moving units of the motion sensing device.

42. A system as claimed in claim 30 wherein the electronics is adapted to generate graphic lines representing relative difference in position between the virtual image of the student'"'"'s movement and the virtual image of the teacher'"'"'s movement;

the lines changing in length as the difference changes.

43. A system as claimed in claim 30 wherein the electronics is adapted to generate an audio tone representing the relative difference in position between the virtual image of the student'"'"'s movement and the virtual image of the teacher'"'"'s movement, the audio tone changing in frequency as the difference changes.

44. A system as claimed in claim 30 wherein the electronics is adapted to generate an intermediate teacher'"'"'s movement by interpolating between previous student'"'"'s movements and the teacher'"'"'s movement.

45. A method for teaching motion, comprising:

sensing a student'"'"'s movement with a motion sensing device;

displaying a student'"'"'s virtual image sequence in response to the student'"'"'s movement in real-time; and

displaying a virtual image sequence of a teacher'"'"'s movement form memory with the student'"'"'s virtual image sequence, such that the student may cause the student'"'"'s virtual image sequence to follow the teacher'"'"'s virtual image sequence.

46. The method of claim 45, wherein the step of sensing a student'"'"'s movement further comprises sensing the position and orientation of an implement set in motion by the student'"'"'s movement.

47. The method of claim 46, wherein the implement is a racquet.

48. The method of claim 46, further comprising the steps of striking a projectile with the implement set in motion by the student and displaying the resulting trajectory of the projectile.

49. The method of claim 48, wherein the step of displaying the trajectory of the projectile takes into account the projectile'"'"'s spin.

50. The method of claim 46 wherein the steps of displaying further comprises displaying an enlarged view of the implement.

51. The method of claim 46 wherein the step of displaying further comprises the step of displaying a shadow of the implement.

52. The method of claim 45, wherein the step of displaying an image sequence of the teacher'"'"'s movement from memory includes prestoring an acceptable movement of the student as the teacher'"'"'s movement.

53. The method of claim 45, wherein the step of displaying the image sequence of the teacher'"'"'s movement from memory incudes displaying the teacher'"'"'s movement at a reduced speed.

54. The method of claim 53, further comprising the step of increasing the speed of the teacher'"'"'s movement as the student becomes proficient at following the teacher'"'"'s movement.

55. The method of claim 45 wherein the step of displaying an image sequence of a teacher'"'"'s movement from memory includes the step of selecting a teacher'"'"'s movement from a library of stored teacher'"'"'s movements according to the student'"'"'s needs.

56. The method of claim 45 wherein the step of sensing a student'"'"'s movement with a motion sensing device further comprises the step of establishing an oriented electromagnetic field between fixed and moving units of the motion sensing device.

57. The method of claim 45 further comprising the step of generating graphic lines representing relative difference in position between the virtual image of the student'"'"'s movement and the virtual image of the teacher'"'"'s movement;

the lines changing in length as the difference changes.

58. The method of claim 45 further comprising the step of generating an audio tone representing the relative difference in position between the virtual image of the student'"'"'s movement and the virtual image of the teacher'"'"'s movement, the audio tone changing in frequency as the difference changes.

59. The method of claim 45 further comprising the step of generating an intermediate teacher'"'"'s movement by interpolating between previous student'"'"'s movements and the teacher'"'"'s movement.

60. A teaching system comprising:

a motion sensing device for sensing student'"'"'s movement;

a display; and

electronics responsive to the motion sensing device to cause a student'"'"'s virtual image sequence on the display in real-time corresponding to a student'"'"'s movement, and to simultaneously cause a teacher'"'"'s virtual image sequence on the display corresponding to a teacher'"'"'s movement such that the student may cause the student'"'"'s movement to follow the teacher'"'"'s movement.

61. A teaching system comprising:

a motion sensing device for sensing the position and orientation of a member set in a motion by a student;

a display;

a memory; and

electronics responsive to the motion sensing device to generate, in real-time, a student'"'"'s virtual image sequence on the display derived from the sensed position and orientation of the member and simultaneously generating a teacher'"'"'s virtual image sequence on the display of a corresponding teacher'"'"'s movement from memory such that the student may cause the student'"'"'s virtual image sequence to follow the teacher'"'"'s virtual image sequence;

the student'"'"'s and teacher'"'"'s virtual image sequences being endpoint trajectories of the member in motion without images of the student and teacher.

62. A method for teaching motion, comprising the steps of:

sensing the position and orientation of a member set in motion by a student with a motion sensing device;

displaying a student'"'"'s virtual image sequence as an endpoint trajectory derived from the sensed position and orientation of the member in real-time; and

displaying a teacher'"'"'s virtual image sequence as an endpoint trajectory of a teacher'"'"'s movement, from memory, with the student'"'"'s virtual image sequence, such that the student may cause the student'"'"'s virtual image sequence to imitate the motion of the teacher'"'"'s virtual image sequence.