Method and system for tracking motion-sensing device

• US 9,352,230 B1
• Filed: 05/11/2011
• Issued: 05/31/2016
• Est. Priority Date: 07/14/2006
• Status: Active Grant

• Current Assignee: Wei Yen
• Original Assignee: AiLive, Inc.
• Inventors: Wilkinson, Dana, Zhu, Youding, Musick, JR., Charles, Powers, III, William Robert
• Primary Examiner(s): Deodhar, Omkar
• Assistant Examiner(s): LI, WEI
• Application Number:
  US13/105,863
• Time in Patent Office: 1,847 Days
• Field of Search: 463/36, 463/37
• US Class Current: 1/1
• CPC Class Codes: A63F 13/00 Video games, i.e. games using an electronically generated display having two or more dimensions gaming systems which provide a financial reward G07F17/32
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 2300/105 using inertial sensors, e.g. accelerometers, gyroscopes
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands

Abstract

Techniques for interpreting motions of a motion-sensitive device are described to allow natural and intuitive interfaces for controlling an application (e.g. video game). A motion-sensitive device includes inertial and generates sensor signals sufficient to derive positions and orientations of the device in six degrees of freedom. The motion of the device in six degrees of freedom is tracked by analyzing sensor data from the inertial sensors in conjunction with data from a secondary source that may be a camera or a non-inertial sensor. Different techniques are provided to correct or minimize errors in deriving the positions and orientations of the device. These techniques include “stop” detection, back tracking, extrapolation of sensor data beyond the sensor ranges, and using constraints from multi trackable objects in an application being interfaced with the motion-sensitive device.

Claims

1. A method for determining tracking a motion-sensing device in six degrees of freedom, the method comprising:

receiving sensor data sequentially from inertial sensors in the motion-sensing device;

estimating three relative angular and three relative translational motions of the motion-sensing device from the sensor data; and

correcting the three relative angular and three relative translational motions of the motion-sensing device to derive a relative motion path of the motion-sensing device, wherein said correcting the relative three angular and three translational motions of the motion-sensing device comprises;

detecting when any of the sensor data has been maxed out because of the motion-sensing device being moved beyond limits of the inertial sensors;

estimating overflow values for a portion of the sensor data that has been maxed out; and

replacing the portion of the sensor data that has been maxed out with the overflow values.

2. The method as recited in claim 1, further comprising:

detecting when the motion-sensing device has been moved with reference to a threshold; and

adjusting the threshold to lower a predefined sensitivity so that a movement of the motion-sensing device is detectable.

3. The method as recited in claim 2, further comprising updating parameters to be used to estimate the three relative angular and relative translational motions of the motion-sensing device when the movement of the motion-sensing device is detected, the parameters including a portion of the sensor data and the adjusted threshold.

4. The method as recited in claim 1, further comprising:

maintaining a recent history of all movements of the motion-sensing device;

initiating to track the motion-sensing device when a motion of the motion-sensing device is detected exceeding a threshold; and

including retroactively a part of the history in an initial moment of tracking the motion-sensing device;

or

removing retroactively a part of the history from quiescence of the motion-sensing device being detected.

5. The method as recited in claim 1, further comprising:

storing a beginning portion of the sensor data when a motion from the beginning portion of the sensor data is not detected in reference to a threshold; and

fitting the beginning portion of the sensor data retroactively to a remaining portion of the sensor data after a motion from the remaining portion of the sensor data is detected in reference to the threshold.

6. The method as recited in claim 1, wherein said estimating overflow values for a portion of the sensor data that has been maxed out comprises:

determining a function that is closely represents a portion of the sensor data has been not maxed out; and

extrapolating the overflow values from the function.

7. The method as recited in claim 6, wherein said correcting the relative three angular and three translational motions of the motion-sensing device comprises extrapolating new values that correspond to the portion of the sensor data that has been maxed out to make the inertial sensors in the motion-sensing device to have a greater dynamic range.

8. The method as recited in claim 1, wherein said correcting the three relative angular and three relative translational motions of the motion-sensing device comprises:

detecting a stop, where an abrupt direction changes in a movement in the sensor data; and

concatenating two motions before and after the stop to recover the movement of the motion-sensing device.

9. The method as recited in claim 1, further comprising:

receiving signals from a secondary source monitoring the motion-sensing device when the sensor data is produced from the motion-sensing device,

determining coordinates of the three relative angular and three relative translational motions of the motion-sensing device with the signals from the secondary source; and

determining three absolute angular and three absolute translational motions of the motion-sensing device to derive a complete motion tracking of the motion-sensing device.

10. The method as recited in claim 9, wherein the secondary source is at least one camera, and the signals are image data, and said determining coordinates of the three relative angular and three relative translational motions comprises:

determining consecutive points in a three-dimensional space representing the three relative angular motions and three relative translational motions of the motion-sensing device; and

determining consecutive points in a two-dimensional space from the image data, these consecutive points in the two-dimensional space corresponding to the consecutive points in the three-dimensional space.

11. The method as recited in claim 10, further comprising:

determining a difference in yaw between the three-dimensional space and the two-dimensional space; and

estimating a depth of each of the consecutive points in the two-dimensional space, resulting in a set of consecutive points in a three-dimensional space determined by the camera.

12. The method as recited in claim 11, further comprising:

synthesizing a new set of consecutive points that incorporate much less error than the consecutive points in the three-dimensional space, and the consecutive points in the two-dimensional space.

13. The method as recited in claim 9, wherein one part of the sensor data is from inertial sensors in the motion-sensing device, and the secondary source generates a local sensible field, another part of the sensor data is from a non-inertial sensor in the motion-sensing device to sense the local sensible field to facilitate determination of the three relative angular and three relative translational motions of the motion-sensing device.

14. The method as recited in claim 13, wherein the local sensible field is a magnetic field, and the non-inertial sensor in the motion-sensing device is a magnetometer.

15. The method as recited in claim 1, wherein the motion-sensing device is being used by a user to interact with an application being displayed, there are multiple objects in the application being tracked, and said correcting the three relative angular and three relative translational motions of the motion-sensing device is achieved by combining constraints from the multiple trackable objects.

16. The method as recited in claim 15, wherein there is another motion-sensing device, at least two of the multiple trackable objects are being tracked by the two motion-sensing devices, one of the constraints being a relationship between two respective references for the two motion-sensing devices, wherein the relationship is being built up over time as the user continues to interact with the application.

17. A computing device for determining tracking a motion-sensing device in six degrees of freedom, the computing device comprising:

a transceiver receiving sensor data sequentially from inertial sensors in the motion-sensing device;

a memory space;

a processor coupled to the memory space; and

a data processing unit, coupled to the memory space and controlled by the processor, configured to perform operations of;

estimating three relative angular and three relative translational motions of the motion-sensing device from the sensor data; and

correcting the three relative angular and three relative translational motions of the motion-sensing device to derive a relative motion path of the motion-sensing device, wherein said correcting the relative three angular and three translational motions of the motion-sensing device comprises

detecting when any of the sensor data has been maxed out because of the motion-sensing device being moved beyond limits of the inertial sensors;

estimating overflow values for a portion of the sensor data that has been maxed out; and

replacing the portion of the sensor data that has been maxed out with the overflow values.

18. The computing device as recited in claim 17, wherein the operations further include:

detecting when the motion-sensing device has been moved with reference to a threshold; and

adjusting the threshold to lower a predefined sensitivity so that a movement of the motion-sensing device is detectable.

19. The computing device as recited in claim 18, wherein the operations further include:

maintaining a recent history of all movements of the motion-sensing device in the memory space;

initiating to track the motion-sensing device when a motion of the motion-sensing device is detected exceeding the adjusted threshold; and

including retroactively a part of the history in an initial moment of tracking the motion-sensing device;

or

removing retroactively a part of the history from quiescence of the motion-sensing device being detected.

20. The computing device as recited in claim 18, wherein the operations further include:

storing a beginning portion of the sensor data in the memory space when a motion from the beginning portion of the sensor data is not detected in reference to the threshold; and

fitting the beginning portion of the sensor data retroactively to a remaining portion of the sensor data after a motion from the remaining portion of the sensor data is detected in reference to the threshold.

21. The computing device as recited in claim 17, wherein the operations further include:

determining a function that is closely represents a portion of the sensor data has been not maxed out; and

extrapolating the overflow values from the function.

22. The computing device as recited in claim 17, wherein the operations further include extrapolating new values that correspond to the portion of the sensor data that has been maxed out to make the inertial sensors in the motion-sensing device to have a greater dynamic range.

23. The computing device as recited in claim 17, wherein the operations further include:

detecting a stop, where an abrupt direction changes in a movement in the sensor data; and

concatenating two motions before and after the stop to recover the movement of the motion-sensing device.

24. The computing device as recited in claim 17, wherein the operations further include:

receiving signals from a secondary source monitoring the motion-sensing device when the sensor data is produced from the motion-sensing device,

determining coordinates of the three relative angular and three relative translational motions of the motion-sensing device with the signals from the secondary source; and

determining three absolute angular and three absolute translational motions of the motion-sensing device to derive a complete motion tracking of the motion-sensing device.

25. The computing device as recited in claim 24, wherein the secondary source is at least one camera, and the signals are image data, and the operations further include:

determining consecutive points in a three-dimensional space representing the three relative angular motions and three relative translational motions of the motion-sensing device; and

determining consecutive points in a two-dimensional space from the image data, these consecutive points in the two-dimensional space corresponding to the consecutive points in the three-dimensional space.

26. The computing device as recited in claim 25, wherein the operations further include:

determining a difference in yaw between the three-dimensional space and the two-dimensional space; and

estimating a depth of each of the consecutive points in the two-dimensional space, resulting in a set of consecutive points in a three-dimensional space determined by the camera.

27. The computing device as recited in claim 26, wherein the operations further include:

synthesizing a new set of consecutive points that incorporate much less error than the consecutive points in the three-dimensional space, and the consecutive points in the two-dimensional space.

28. The computing device as recited in claim 17, wherein one part of the sensor data is from inertial sensors in the motion-sensing device, and the secondary source generates a local sensible field, another part of the sensor data is from a non-inertial sensor in the motion-sensing device to sense the local sensible field to facilitate determination of the three relative angular and three relative translational motions of the motion-sensing device.

29. The computing device as recited in claim 18, wherein the local sensible field is a magnetic field, and the non-inertial sensor in the motion-sensing device is a magnetometer.

30. The computing device as recited in claim 17, wherein the motion-sensing device is being used by a user to interact with an application being displayed, there are multiple objects in the application being tracked, and said correcting the three relative angular and three relative translational motions of the motion-sensing device is achieved by combining constraints from the multiple trackable objects.

31. The computing device as recited in claim 30, wherein there is another motion-sensing device, at least two of the multiple trackable objects are being tracked by the two motion-sensing devices, one of the constraints being a relationship between two respective references for the two motion-sensing devices, wherein the relationship is being built up over time as the user continues to interact with the application.

32. A motion-sensing device being tracked in six degrees of freedom, the motion-sensing device comprising:

a set of inertial sensors that generate a first set of sensor data when the motion-sensing device is being manipulated by a user to interact with an application; and

a transceiver to transport the first set of sensor data to a computing device that is configured to derive a relative moving path in 6 axes including three relative angular and three translational motions of the motion-sensing device, wherein the three relative angular and three relative translational motions of the motion-sensing device are corrected to derive a relative motion path of the motion-sensing device by;

detecting when any of the sensor data has been maxed out because of the motion-sensing device being moved beyond limits of the inertial sensors;

estimating overflow values for a portion of the sensor data that has been maxed out; and

replacing the portion of the sensor data that has been maxed out with the overflow values.

33. The motion-sensing device as recited in claim 32, further comprising at least a non-inertial sensor that produces a second set of sensor data that is also transported via the transceiver therein and used to get at least one of the 6 axes as a known parameter so that less than the 6 axes need to be computed by the computing device from the first set of sensor data.

34. The motion-sensing device as recited in claim 33, wherein the non-inertial sensor produces the second set of sensor data when the motion-sensing device is being moved in a sensible field.

35. The motion-sensing device as recited in claim 34, wherein the local sensible field is a magnetic field, and the non-inertial sensor in the motion-sensing device is a magnetometer.

36. The motion-sensing device as recited in claim 32, wherein the motion-sensing device carries at least a mark, a camera producing a sequence of images capturing movements of the mark to facilitate the computing device to determine an absolute motion tracking of the motion-sensing device from the first set of sensor data together with the image data.

37. The motion-sensing device as recited in claim 36, wherein the camera is disposed around a user holding the motion-sensing device.