Self-Contained Inertial Navigation System for Interactive Control Using Movable Controllers

• US 20100113153A1
• Filed: 01/25/2008
• Published: 05/06/2010
• Est. Priority Date: 07/14/2006
• Status: Active Grant

• Current Assignee: Wei Yen
• Original Assignee: AiLive, Inc.
• Inventors: Reynolds, Stuart, Tu, Xiaoyuan, Funge, John, Bererton, Curt, Wright, Ian, Yen, Wei, Dobson, Daniel, Musick, Charles, Powers, William Robert III
• Granted Patent: US 9,405,372 B2
• Time in Patent Office: Days
• US Class Current: 463/37
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/212 using sensors worn by the player, e.g. for measuring heart beat or leg activity
• CPC Class Codes: A63F 13/235 using a wireless connection, e.g. infrared or piconet
• CPC Class Codes: A63F 13/24 Constructional details thereof, e.g. game controllers with detachable joystick handles
• CPC Class Codes: A63F 13/42 by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 2300/1012 involving biosensors worn by the player, e.g. for measuring heart beat, limb activity
• CPC Class Codes: A63F 2300/1031 using a wireless connection, e.g. Bluetooth, infrared connections
• CPC Class Codes: A63F 2300/1043 being characterized by constructional details
• CPC Class Codes: A63F 2300/105 using inertial sensors, e.g. accelerometers, gyroscopes
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors

Abstract

A movable game controller for controlling aspects of a computer controlled game display with apparatus for determining the linear and angular motion of that movable controller. The apparatus includes a plurality of self-contained inertial sensors for sensing the tri-axial linear and tri-axial angular motion of the moving controller. Each sensor is mounted at a fixed linear position and orientation with respect to the others. The linear and angular motion of the controller is computed from the correlated motion sensor readings of each of the plurality of self-contained inertial sensors.

Claims

1. A method, including steps of

in response to (a) a time series of data responsive to a set of sensors, that time series of data including information about one or more components of the time-varying position and orientation of a movable object in a multi-dimensional space, and (b) a set of assumptions regarding that object, that object being located in a terrestrial frame of reference, being substantially movable by human motive force, and being at least one of:

handheld, wearable, attachable to a living creature,

determining a set of estimated errors associated with information included in that time series; and

in response to that time series and that set of estimated errors, determining a set of estimates of time-varying changes in one or more of components of position and orientation of that object.

2. A method as in claim 1, wherein

that time series includes information sufficient to track changes in all six components of position and orientation of that object in a three-dimensional space, and

that set of estimates of time-varying changes includes estimates of time-varying changes in all six components of position and orientation of that object.

3. A method as in claim 1, wherein

that time series includes information insufficient to track changes in all six components of position and orientation of that object in a three-dimensional space;

that set of assumptions, in combination with that time series, includes information sufficient to track changes in all six components of position and orientation of that object in a three-dimensional space; and

that set of estimates of time-varying changes includes estimates of time-varying changes in all six components of position and orientation of that object.

4. A method as in claim 1, wherein

that set of estimated errors relates to errors in outputs from those sensors.

5. A method as in claim 1, wherein

that set of estimated errors relates to errors in that time series of data.

6. A method as in claim 1, wherein

those sensors are self-contained inertial sensors.

7. A method as in claim 6, wherein

those steps of determining a set of estimates of time-varying changes operate in substantially real-time relative to that time-varying position and orientation of that object.

8. A method as in claim 6, wherein

those steps of determining a set of estimated errors are responsive to a set of sensors other than self-contained inertial sensors.

9. A method as in claim 8, wherein

those sensors other than self-contained inertial sensors include at least one of

an infra-red sensor,

an optical recognizer,

a sonic device,

an electromagnetic field sensor,

a device capable of determining information from triangulation,

a GPS receiver.

10. A method as in claim 6, including steps of

identifying one or more periods of time in which that object is substantially quiescent.

11. A method as in claim 10, wherein

those steps of identifying one or more periods of time include steps of

identifying subsequences of that time series in which the observed acceleration of that object can be substantially attributed to gravity alone.

12. A method as in claim 10, wherein

data for one or more of those periods of quiescence includes information sufficient to determine a direction of gravity.

13. A method as in claim 10, including steps of

determining an orientation of that object for a period of quiescence in response to data from a set of sensors including at least one sensor other than a self-contained inertial sensor.

14. A method as in claim 10, wherein

data for that time of quiescence includes information responsive to a direct pointing device.

15. A method as in claim 10, wherein

estimates for changes in one or more components of the position and orientation of the object are only provided for a subset of the time series starting with a period of quiescence for the object.

16. A method as in claim 6, including steps of

maintaining that set of estimated errors for use by those steps of determining a set of estimates of changes in one or more components of position and orientation in response to one or more later time-varying changes in one or more components of position and orientation of that object.

17. A method as in claim 6, including steps of

maintaining that set of estimated errors for use by those steps of determining a set of estimates of changes in one or more components of position and orientation in response to one or more earlier time-varying changes in one or more components of position and orientation of that object.

18. A method as in claim 6, including steps of

presenting an animation in response to that set of changes in one or more components of position and orientation of that object.

19. A method as in claim 6, wherein

those steps of determining a set of estimated errors and determining a set of estimates of time-varying changes

are performed iteratively in response to receipt of further data in that time series.

20. A method as in claim 6, including steps of

determining a set of re-estimated errors in response to that set of estimated errors, in response to information obtained after that set of estimated errors;

and

in response to that time series and that set of re-estimated errors, determining a revised set of estimates of configuration of that object.

21. A method as in claim 6, including steps of

determining a tentative set of estimates of configuration of that object;

wherein those steps of determining a set of estimates are responsive to that tentative set of estimates.

22. A method as in claim 6, wherein

those assumptions include information regarding constraints on possible configurations of that movable object.

23. A method as in claim 6, wherein

those assumptions include information regarding constraints on possible information appearing in that time series of data.

24. A method as in claim 6, wherein

that set of estimates of time varying changes in one or more components of the orientation of that object is responsive to a first computation;

that set of estimates of time varying changes in one or more components of position of that object is responsive to a second computation;

that first computation being substantially independent of that second computation.

25. A method as in claim 6, wherein

that set of constraints is responsive to domain knowledge about at least one of

a typical use of that object,

a typical location of use of that object,

a biomechanical feature of a user of that object,

a motion recognition signal associated with that time series of data.

26. A method as in claim 6, wherein

that set of constraints is responsive to domain knowledge about a particular application of that object;

that domain knowledge about a particular application includes at least one of

within that particular application, one or more typical motions of that object,

within that particular application, a set of unlikely motions of that object.

27. A method as in claim 26, wherein

that particular application of that object includes a possibly-fictional world.

28. A method as in claim 27, wherein

that particular application of that object includes a set of rules applicable to that possibly-fictional world.

29. A method as in claim 6, wherein

additional information regarding possible configurations for the object is provided by an associated application interactive with that object.

30. A method as in claim 29, wherein the associated application provides bounds on one or more of

components of the allowable positions, and single and multiple derivatives thereof;

components of the allowable orientations, and single and multiple derivatives thereof.

31. A method as in claim 29, wherein

that additional information includes one or more components of an initial position and orientation for that object.

32. A method as in claim 29, wherein

that additional information includes one or more components of position and orientation for that object known at a selected period of time.

33. A method, including steps of

in response to a time series of data responsive to a set of self-contained inertial sensors, that time series of data including information about one or more components of the time-varying position and orientation of a movable object in a multi-dimensional space, that object being located in a terrestrial frame of reference, being substantially movable by human motive force, and being at least one of:

handheld, wearable, attachable to a living creature,

determining a set of estimates of time-varying changes in one or more of components of position and orientation of that object,

providing a model of a multi-dimensional virtual environment,

in response to that set of estimates and that model, calculating a point and direction of interest in that model.

34. A method as in claim 33, wherein

that multi-dimensional space includes at least three dimensions.

35. A method as in claim 34, wherein

that time series includes information sufficient to track changes in all six components of position and orientation of that object in a three-dimensional space, and

that method includes steps to determine a set of estimates of time-varying changes in all six components of position and orientation of that object.

36. A method, as in claim 34, wherein

that point and direction of interest are used to select a virtual object in that model.

37. A method, as in claim 36, wherein

that virtual object is chosen by following a ray from the point of interest along the direction of interest and selecting the first object it intersects with in that model.