System and Method for Sensor Fusion of Single Range Camera Data and Inertial Measurement for Motion Capture

• US 20130222565A1
• Filed: 02/28/2013
• Published: 08/29/2013
• Est. Priority Date: 02/28/2012
• Status: Abandoned Application

• Current Assignee: Johns Hopkins University
• Original Assignee: Johns Hopkins University
• Inventors: Guerin, Kelleher Riccio, Okamura, Allison, Rotella, Michele, He, Xingchi, Bohren, Jonathan
• Application Number:
  US13/780,252
• Publication Number:
  US 20130222565A1
• Time in Patent Office: Days
• US Class Current: 348/77
• CPC Class Codes: A61B 2562/0219 Inertial sensors, e.g. accelerometers, gyroscopes, tilt switches
• CPC Class Codes: A61B 5/11 Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb for measuring pulse A61B5/02 ;A61B5/1038 takes precedence; motion detection to correct for motion artifacts in physiological signals A61B5/721
• CPC Class Codes: A61B 5/1128 using image analysis A61B5/1127 takes precedence
• CPC Class Codes: A61B 5/6831 Straps, bands or harnesses
• CPC Class Codes: A61B 5/744 Displaying an avatar, e.g. an animated cartoon character
• CPC Class Codes: A61B 5/7455 characterised by tactile indication, e.g. vibration or electrical stimulation
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 2300/105 using inertial sensors, e.g. accelerometers, gyroscopes
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: G06F 18/251 of input or preprocessed data
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06V 10/803 of input or preprocessed data
• CPC Class Codes: G06V 40/23 : Recognition of whole body movements, e.g. for sport training

Abstract

An embodiment in accordance with the present invention provides a motion capture system that can be coupled with a feedback system. The system can be used for collecting motion capture data about a user'"'"'s position and/or movement and comparing that data to an exemplary position or movement to determine errors in the position or movement. The system can include a single range camera to provide color and depth data as well as inertial sensors to provide inertial measurements. Additionally, the system can provide feedback to the user to encourage correction of the errors the system finds with respect to the user'"'"'s position and/or movement.

Claims

1. A system for motion capture of a user comprising:

a single range camera configured and positioned to collect data;

inertial sensors configured and positioned to collect inertial motion data;

a computer readable medium configured to analyze the data from the single range camera and the inertial motion data from the inertial sensors in order to determine a position of the user.

2. The system for motion capture of the user of claim 1, wherein the computer readable medium is further programmed to compare the position of the user to data for an exemplary position.

3. The system for motion capture of the user of claim 2, wherein the data for an exemplary position comprises data from one of a teacher also using the system for motion capture and predetermined data for the exemplary position stored on a computer readable medium.

4. The system for motion capture of the user of claim 3, wherein the computer readable medium is further programmed to provide a calculation of error between the position of the user and the data for the exemplary position.

5. The system for motion capture of the user of claim 4 further comprising a device for providing feedback to the user to correct the error between the position of the user and the data for the exemplary position.

6. The system for motion capture of the user of claim 5 wherein the device for providing feedback to the user comprises at least one of visual feedback, tactile feedback, vibrotactile feedback, and sound feedback.

7. The system for motion capture of claim 5 wherein the device for providing feedback to the user takes the form of an eccentric mass motor.

8. The system for motion capture of claim 1 wherein the inertial sensor takes the form of an accelerometer.

9. The system for motion capture of claim 8 wherein the accelerometer further takes the form of a 3-axis accelerometer.

10. The system for motion capture of claim 1 wherein the inertial sensor is positioned on a body part of the user for which it is to collect inertial motion data.

11. The system for motion capture of claim 1 wherein the inertial sensors are positioned to collect inertial motion data for a whole body of the user.

12. The system for motion capture of claim 1 wherein the inertial sensors are mounted in bands to allow the inertial sensors to be secured to the user.

13. The system for motion capture of claim 1 wherein the inertial sensors are positioned at a distal-most point of a limb-segment of the user.

14. The system for motion capture of claim 1 further comprising a visual display.

15. The system for motion capture of claim 14 wherein the visual display is further configured to show the user a visual representation of the user'"'"'s movement.

16. The system for motion capture of claim 15 wherein the visual display is further configured to show the user a visual representation of the user'"'"'s movement relative to an exemplary movement.

17. The system for motion capture of claim 1 further comprising a microcontroller.

18. The system for motion capture of claim 17 wherein the microcontroller is configured to control the inertial sensors as well as feedback to the user.

19. A method for motion capture of a user comprising:

collecting data related to the user'"'"'s motion using a single range camera;

collecting inertial motion data for the user'"'"'s motion; and

analyzing the data from the single range camera and the inertial motion data in order to determine a position of the user.

20. The method of claim 19 further comprising comparing the position of the user to data for an exemplary position.

21. The method of claim 20 further comprising calculating error between the position of the user and the data for the exemplary position

22. The method of claim 19 further comprising providing feedback to the user to correct the error between the position of the user and the data for the exemplary position.

23. The method of claim 19 further comprising providing visual feedback to the user.