Motion tracking system and method using camera and non-camera sensors

• US 8,696,458 B2
• Filed: 02/15/2008
• Issued: 04/15/2014
• Est. Priority Date: 02/15/2008
• Status: Active Grant

• Current Assignee: Thales Visionix Incorporated (Thales SA)
• Original Assignee: Thales Visionix Incorporated (Thales SA)
• Inventors: Foxlin, Eric, Wormell, Dean, Browne, Thomas C., Donfrancesco, Michael
• Primary Examiner(s): GALKA, LAWRENCE STEFAN
• Application Number:
  US12/032,079
• Publication Number:
  US 20090209343A1
• Time in Patent Office: 2,251 Days
• Field of Search: 463/36, 345/156, 382/103
• US Class Current: 463/36
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G07F 17/32 for games, toys, sports, or amusements
• CPC Class Codes: G07F 17/3209 Input means, e.g. buttons, touch screen

Abstract

Systems and methods for tracking the motion of a game controller and a player'"'"'s body part are disclosed. The method includes receiving a video signal from a webcam, generating movement data corresponding to the motion of the game controller, tracking first predetermined degrees of freedom of motion of the body part, generating feature location data based on a location of at least one identifiable visual feature of the game controller, and estimating second predetermined degrees of freedom of a motion of the game controller.

Claims

1. A system for tracking the motion of an input device held in or attached to a user'"'"'s first body part and a user'"'"'s second body part that is movable independently of said first body part, the system comprising:

a motion sensor coupled to the input device, the sensor configured to generate movement data indicative of movement of the input device,

a computer-vision module adapted to process video signals from a camera, and in response to the video signals, and without relying on any data derived from an electronic device held in or attached to the second body part, to track motion of the second body part,

an image-processing module adapted to process the video signals, and in response to the video signals, to generate feature location data based on a location of at least one identifiable visual feature on the input device, and

a data-fusion module adapted to receive the feature location data from the image-processing module, and the movement data from the sensor, and track motion of the input device.

2. The system of claim 1 in which the motion of the second body part includes at least two degrees of freedom.

3. The system of claim 1 in which the motion of the input device includes at least two degrees of freedom.

4. The system of claim 1 in which the sensor is installed in the input device.

5. The system of claim 1 in which the image-processing module is configured to identify a location in an image of the at least one identifiable visual feature on the input device.

6. The system of claim 1 in which the sensor includes at least one accelerometer.

7. The system of claim 1 in which the sensor includes at least one angular rate sensor.

8. The system of claim 1 in which the sensor includes at least one magnetometer.

9. The system of claim 1 in which the sensor includes a three-axis linear accelerometer.

10. The system of claim 1 in which the sensor includes a three-axis angular rate sensor.

11. The system of claim 1 in which a tracked motion of a player'"'"'s head is used to control the viewpoint for rendering graphics in a game.

12. The system of claim 1 in which a tracked motion of the second body part is used to control a virtual representation of the second body part in a game.

13. The system of claim 1 in which a tracked motion of the second body part is used to control the player'"'"'s locomotion or navigation through a game world.

22. The system of claim 1 in which the second body part is one of the user'"'"'s head, the user'"'"'s face, one or both of the user'"'"'s eyes, or an object worn on the user'"'"'s head or face, and the input device is held in one of the user'"'"'s hands.

23. The system of claim 22 in which the input device is a game controller and the user is a game player.

24. The system of claim 22 in which the input device is a wand used to control an immersive visualization environment.

14. A method performed by an information handling system comprising:

receiving video signals from a camera, the video signals based on images of at least one of an input device held in or attached to a user'"'"'s first body part and a second body part of a user that is movable independently of said first body part,

using a motion sensor coupled to the input device to generate movement data, the movement data corresponding to the motion of the input device,

using a computer vision module to process video signals from the camera to track motion of the second body part without relying on any data derived from an electronic device held in or attached to the second body part, and using an image processing module to process the video signals to generate feature location data based on a location of at least one identifiable visual feature on the input device, and

using a data-fusion module to track motion of the input device based on the movement data, and the feature location data.

15. The method of claim 14 further comprising calibrating the camera to have a view of at least one of the input device and the second body part.

16. The method of claim 14 in which the motion of the body part has at least two degrees of freedom.

17. The method of claim 14 in which the motion of the input device has at least two degrees of freedom.

18. The method of claim 14 in which the at least one identifiable visual feature is located on the input device.

19. The method of claim 14 in which the second body part is at least one of the player'"'"'s head, one of the player'"'"'s arms, one of the player'"'"'s feet, one of the player'"'"'s hands, and one of the player'"'"'s eyes.

20. A non-transitory computer-readable medium comprising instructions to cause a data processing system to:

receive video signals from a camera, the video signals based on images of at least one of an input device held in or attached to a user'"'"'s first body part and a second body part of a user that is movable independently of said first body part,

generate movement data, the movement data corresponding to the motion of the input device,

in response to the video signals, track motion of the second body part without relying on any data derived from an electronic devices held in or attached to the second body part, and generate feature location data based on a location of at least one identifiable visual feature, and

based on the movement data, and the feature location data, track motion of the input device.

21. The medium of claim 20 further comprising instructions to cause the input device to calibrate the camera to have a view of at least one of the input device and the second body part.