Self-contained inertial navigation system for interactive control using movable controllers

• US 7,636,645 B1
• Filed: 06/18/2007
• Issued: 12/22/2009
• Est. Priority Date: 06/18/2007
• Status: Active Grant

• Current Assignee: Atlive, Inc.
• Original Assignee: AiLive, Inc.
• Inventors: Bererton, Curt, Dobson, Daniel, Funge, John, Musick, Charles, Powers, III, William Robert, Reynolds, Stuart, Tu, Xiaoyuan, Wright, Ian, Yen, Wei
• Primary Examiner(s): Feliciano, Eliseo Ramos
• Assistant Examiner(s): Suarez, Felix E
• Application Number:
  US11/820,207
• Time in Patent Office: 918 Days
• Field of Search: 702150-152, 702/156, 702/158, 702/182, 702/189, 702/197, 702/153, 702/179, 463/37, 600/587, 700/245
• US Class Current: 702/152
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/235 using a wireless connection, e.g. infrared or piconet
• CPC Class Codes: A63F 13/24 Constructional details thereof, e.g. game controllers with detachable joystick handles
• CPC Class Codes: A63F 13/833 Hand-to-hand fighting, e.g. martial arts competition A63F13/837 takes precedence
• CPC Class Codes: A63F 13/92 Video game devices specially adapted to be hand-held while playing
• CPC Class Codes: A63F 13/98 Accessories, i.e. detachable arrangements optional for the use of the video game device, e.g. grip supports of game controllers
• CPC Class Codes: A63F 2300/1006 having additional degrees of freedom
• CPC Class Codes: A63F 2300/1031 using a wireless connection, e.g. Bluetooth, infrared connections
• CPC Class Codes: A63F 2300/1043 being characterized by constructional details
• CPC Class Codes: A63F 2300/105 using inertial sensors, e.g. accelerometers, gyroscopes
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: A63F 2300/204 the platform being a handheld device
• CPC Class Codes: A63F 2300/8029 Fighting without shooting
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06N 20/00 Machine learning

Abstract

A movable game controller for controlling aspects of a computer controlled game display with apparatus for determining the linear and angular motion of that movable controller. The apparatus includes a plurality of self-contained inertial sensors for sensing the tri-axial linear and tri-axial angular motion of the moving controller. Each sensor is mounted at a fixed linear position and orientation with respect to the others. The linear and angular motion of the controller is computed from the correlated motion sensor readings of each of the plurality of self-contained inertial sensors.

Claims

1. A method comprising:

augmenting a device by attaching removably a case thereto, the device containing a first set of less than six self-contained inertial sensors;

providing a second set of less than six self-contained inertial sensors housed in the case, outputs of the second set of inertial sensors being coupled to the device externally through a connection, wherein there are at least six of the self-contained inertial sensors in the first set and the second set in total;

providing, in response to correlated outputs from the first and second sets of the self-contained inertial sensors, an augmented time series including sufficient information to track changes in both position and orientation of the device being manipulated by a user in a three-dimensional space.

2. A method as in claim 1, further comprising providing, in response to that augmented time series, a set of tracked changes in both position and orientation of the device in the three-dimensional space.

3. A method as in claim 2, further including combining a first time series responsive to one or more of the first set of sensors with a second time series responsive to one or more of the second set of sensors.

4. A method as in claim 3, wherein the step of combining is responsive to at least one of:

a substantially known or detectable configuration of one or more of the self-contained inertial sensors in the first set with respect to one or more of the self-contained inertial sensors in the second set; and

a substantially known or detectable timing relationship between one or more of the self-contained inertial sensors in the first set with respect to one or more of the self-contained inertial sensors in the second set.

5. A method as in claim 1, including a model being responsive to changes in either position or orientation of an object in the three-dimensional space.

6. A method as in claim 1, wherein one or more of the sensors in the first set and the second set is substantially movable in at least one of position or orientation by a human motive force.

7. A method as in claim 1, wherein the device is at least one of:

handheld, wearable, attachable to a living creature.

8. A method as in claim 1, wherein the device is configurable with at least one of:

a game, video game, video simulation, a medical device, a robotic device, a toy, a communication system, and a navigation system.

9. A method for interacting with a video game being displayed on a display screen, the method comprising:

detecting a first set of less than six first self-contained inertial sensors being enclosed in a first device that is independently operable, the first device being insufficient to track changes in both position and orientation of the first device in a three-dimensional space; and

detecting a second sets of less than six second self-contained inertial sensors enclosed in a second device being operable independently, the second device being insufficient to track changes in both position and orientation of the second device in a three-dimensional space;

restricting that first and second devices into a defined configuration to determine a set of restrictions of movement of the first and second devices, wherein outputs of the second set of inertial sensors in the second device are coupled to the first device via a wireless link to be correlated with the outputs of the first set of inertial sensors in the first device,

wherein for a combination including substantially of (a) a first time series responsive to that first self-contained inertial sensors, (b) a second time series responsive to that second self-contained inertial sensors, and (c) a set of third assumptions being substantially less restrictive than those first assumptions and those second assumptions, that combination includes sufficient information to track changes in both position and orientation of that first and second devices being manipulated by a user in a three-dimensional space to control a character in the video game.

10. A method as in claim 9, further including providing, in response to that sufficient information, a set of tracked changes in both position and orientation of that first and second devices in the three-dimensional space.

11. A method as in claim 10, further comprising combining the first time series responsive to one or more of the first set of sensors with the second time series responsive to one or more of the second set of sensors.

12. A method as in claim 11, wherein the step of combining is responsive to at least one of

a substantially known or detectable configuration of one or more of those first sensors with respect to one or more of those second self-contained inertial sensors;

and

a substantially known or detectable timing relationship between one or more of those first sensors with respect to one or more of those self-contained inertial second sensors,

at least one of (a) that substantially known or detectable configuration, (b) that substantially known or detectable timing relationship, being responsive to those steps of restricting.

13. A method as in claim 9, including a model being responsive to changes in either position or orientation of an object in the three-dimensional space.

14. A method as in claim 9, wherein

one or more of those self-contained inertial sensors is substantially movable in at least one of position or orientation by a human motive force.

15. A method as in claim 9, wherein an object in the three-dimensional space is at least one of:

handheld, wearable, attachable to a living creature.

16. A method as in claim 9, wherein an object in the three-dimensional space is configurable with at least one of:

a game, video game, video simulation, a medical device, a robotic device, a toy, a communication system, and a navigation system.

17. A method for interacting with a video game being displayed on a display screen, the method comprising:

detecting a set of less than six first self-contained inertial sensors enclosed in a device independently operable, the first self-contained inertial sensors being insufficient to concurrently track substantially unconstrained changes in both position and orientation of the device in a three-dimensional space; and

detecting a set of less than six second self-contained inertial sensors enclosed in a separate package, that second self-contained inertial sensors being insufficient to concurrently track substantially unconstrained changes in both the position and the orientation of the device in the three-dimensional space;

restricting the device and the package into a defined configuration that is provided to determine a set of restrictions of movement of the first self-contained inertial sensors and the second self-contained inertial sensors with respect to each other, wherein signals from the second inertial sensors of the separate package are coupled to a port of the device that is configured to correlate signals from the first inertial sensors of the device and the signals from the inertial sensors of the separate package,

wherein a combination of a first time series responsive to the first self-contained inertial sensors and a second time series responsive to the second self-contained inertial sensors includes sufficient information to concurrently track substantially unconstrained changes in both position and orientation of an object in a three-dimensional space.

18. A method as in claim 17, including steps of

providing, in response to that sufficient information, a set of tracked changes in both position and orientation of the object in a three-dimensional space.

19. A method as in claim 18, further comprising combining the first time series responsive to one or more of the first self-contained inertial sensors with the second time series responsive to one or more of the second self-contained inertial sensors.

20. A method as in claim 19, wherein the step of combining is responsive to at least one of

a substantially known or detectable configuration of one or more of the first self-contained inertial sensors with respect to one or more of the second self-contained inertial sensors;

and

a substantially known or detectable timing relationship between one or more of the first self-contained inertial sensors with respect to one or more of the second self-contained inertial sensors,

at least one of (a) the substantially known or detectable configuration, (b) the substantially known or detectable timing relationship, being responsive to the steps of restricting.

21. A method as in claim 17, including a model being responsive to changes in either position or orientation of that object in a three-dimensional space.

22. A method as in claim 17, wherein

one or more of the first and second self-contained inertial sensors is substantially movable in at least one of position or orientation by a human motive force.

23. A method as in claim 17, wherein the object is at least one of:

handheld, wearable, attachable to a living creature.

24. A method as in claim 17, wherein the object is configurable with at least one of:

a game, video game, or video simulation;

a medical device;

a robotic device;

a toy, a communication system, and a navigation system.

25. A method for interacting with a video game being displayed on a display screen, the method comprising:

augmenting a device by attaching removably a case thereto, the device containing a first set of less than six self-contained inertial sensors and being operable independently, the first set of sensors being insufficient by itself to concurrently track substantially unconstrained changes in both position and orientation of the device in a three-dimensional space;

the augmenting including steps of

providing a second set of less than six self-contained inertial sensors housed in the case, a total number of the self-contained inertial sensors in the first set and the second set is at least six;

enforcing substantial restrictions on movement of the device and the case with respect to each other;

providing, in response to those first and second set of self-contained inertial sensors, sufficient information to track substantially unconstrained changes in both position and orientation of that device in the three-dimensional space in response to those first and second sets of sensors and those restrictions on movement.

26. A method as in claim 25, further comprising:

in response to that sufficient information, tracking substantially unconstrained changes in both position and orientation of that object in a three-dimensional space in response to those first and second sets of self-contained inertial sensors and those restrictions on movement.

27. A method as in claim 26, wherein the step of tracking includes combining a first time series responsive to one or more of the first set of sensors with a second time series responsive to one or more of the second set of sensors.

28. A method as in claim 27, wherein the step of combining is responsive to at least one of:

a substantially known or detectable configuration of one or more of the first self-contained inertial sensors with respect to one or more of the second self-contained inertial sensors; and

a substantially known or detectable timing relationship between one or more of the first self-contained inertial sensors with respect to one or more of the second self-contained inertial sensors,

at least one of (a) the substantially known or detectable configuration, (b) the substantially known or detectable timing relationship, being responsive to the step of enforcing substantial restrictions.

29. A method as in claim 25, including a model, the model being responsive to changes in either position or orientation of the object in the three-dimensional space.

30. A method as in claim 25, wherein one or more of the self-contained inertial sensors in the first and second sets is substantially movable in at least one of position or orientation by a human motive force.

31. A method as in claim 25, wherein the device is at least one of:

handheld, wearable, attachable to a living creature.

32. A method as in claim 25, wherein the device is configurable with at least one of:

a game, video game, video simulation, a medical device, a robotic device;

a toy, a communication system, and a navigation system.

33. A method for interacting with a video game being displayed on a display screen, the method comprising

augmenting a device by attaching a case thereto, the device containing a first set of less than six self-contained inertial sensors, the first set of self-contained inertial sensors being insufficient to concurrently track substantially unconstrained changes in both position and orientation of the device in a three-dimensional space;

the step of augmenting including steps of;

providing a second set of less than six self-contained inertial sensors enclosed in the case, a total number of the self-contained inertial sensors in the first set and second set is at least six;

receiving in the device signals from the second set of inertial sensors of the case; and

providing an augmented time series in response to the first and second sets of self-contained inertial sensors, the augmented time series including sufficient information to concurrently track substantially unconstrained changes in both position and orientation of the device being manipulated by a user in a three-dimensional space to control a character in the video game.

34. A method as in claim 33, further comprising providing, in response to the sufficient information, a set of tracked changes in both position and orientation of an object in a three-dimensional space.

35. A method as in claim 34, further comprising combining a first time series responsive to one or more of the first set of self-contained inertial sensors with a second time series responsive to one or more of the second set of self-contained inertial sensors.

36. A method as in claim 35, wherein the step of combining is responsive to at least one of

a substantially known or detectable configuration of one or more of the first self-contained inertial sensors with respect to one or more of the second self-contained inertial sensors;

and

a substantially known or detectable timing relationship between one or more of the first self-contained inertial sensors with respect to one or more of the second self-contained inertial sensors.

37. A method as in claim 33, including a model, the model being responsive to changes in either position or orientation of the object in the three-dimensional space.

38. A method as in claim 33, wherein one or more of the inertial sensors is substantially movable in at least one of position or orientation by a human motive force.

39. A method as in claim 33, wherein the device is at least one of:

handheld, wearable, attachable to a living creature.

40. A method as in claim 33, wherein the device is configurable with at least one of:

a game, video game, video simulation, a medical device, a robotic device;

a toy, a communication system, and a navigation system.

41. A method for interacting with a video game being displayed on a display screen, the method comprising:

augmenting a device by attaching a case thereto, the device including a first set of less than six first self-contained inertial sensors and being unable by itself to concurrently track substantially unconstrained changes in both position and orientation of the device in a three-dimensional space;

providing a second set of less than six second self-contained inertial sensors enclosed in the case, a total number of the first and second self-contained inertial sensors being at least six in total, wherein signals from the second set of inertial sensors in the case are coupled to the device that is configured to correlate signals from the first set of the inertial sensors with the signals of the second set;

providing an augmented time series in response to the set of first and second self-contained inertial sensors, the augmented time series including sufficient information to concurrently track substantially unconstrained changes in both position and orientation of the device being manipulated by a user in a three-dimensional space to control a character in the video game.

42. A method as in claim 41, further comprising providing, in response to the sufficient information, a set of tracked changes in both position and orientation of an object in a three-dimensional space.

43. A method as in claim 42, wherein the step of providing includes steps of combining a first time series responsive to one or more of the first self-contained inertial sensors with a second time series responsive to one or more of the second self-contained inertial sensors.

44. A method as in claim 43, wherein the step of combining is responsive to at least one of:

a substantially known or detectable configuration of one or more of the first self-contained inertial sensors with respect to one or more of the second self-contained inertial sensors; and

a substantially known or detectable timing relationship between one or more of the first self-contained inertial sensors with respect to one or more of the second self-contained inertial sensors.

45. A method as in claim 42, including a model being responsive to changes in either position or orientation of the object in the three-dimensional space.

46. A method as in claim 41, wherein one or more of the first and second set of self-contained inertial sensors is substantially movable in at least one of position or orientation by a human motive force.

47. A method as in claim 41, wherein the device is at least one of:

handheld, wearable, attachable to a living creature.

48. A method as in claim 41, wherein the device is configurable to couple the second set of self-contained inertial sensors to a power source.

49. A method as in claim 41, wherein the device is configurable to couple the first set of self-contained inertial sensors to a communication element.

50. A method as in claim 49, further comprising operating the device to couple the first and second self-contained inertial sensors to a model of position and orientation of the object in the three-dimensional space.

51. A method as in claim 50, wherein the model of position and orientation includes at least one of:

a game, video game, video simulation, a medical device, a robotic device; and

a toy.

52. A method as in claim 41, wherein the device includes sensors other than self-contained inertial sensors.

53. A method as in claim 41, including steps of providing at least one output signal from the device, wherein the step of providing is responsive to at least one of:

a substantially known or detectable configuration of that device with respect to those second sensors, and

a substantially known or detectable timing relationship of with respect to that output signal and those second sensors.

54. A method as in claim 41, wherein the step of providing includes restricting the device to a substantially known or detectable configuration with respect to the second set of self-contained inertial sensors.

55. A method as in claim 41, wherein at least one of the first and second sensors includes at least one of:

an accelerometer, a gyroscope, a compass, an electromagnetic field sensor, a Hall effect sensor, a Doppler sensor, an optical recognizer, and a sonic device.

56. A method comprising:

providing at least six self-contained inertial sensors into a movable controller, the inertial sensors generating sensing signals when the movable controller is being moved, wherein among said at least six self-contained inertial sensors there must be three for linear acceleration along three linear axes and three for determining angular motion;

correlating the sensing signals so that both three-dimensional linear translation and angular orientation of the moving controller is accurately tracked; and

providing a model of a fictional world, the model being responsive to changes in either position or orientation of the movable controller.

57. A method as in claim 56, further comprising:

generating a time series of information in accordance with said at least six self-contained inertial sensors; and

providing, in response to that time series, a set of tracked changes in both position and orientation of the movable controller in a three-dimensional space.

58. A method as in claim 57, further comprising combining a first time series responsive to a first set of the self-contained inertial sensors with a second time series responsive to a second set of the self-contained inertial sensors.

59. A method as in claim 58, wherein said combining is responsive to at least one of:

a substantially known or detectable configuration of one or more of the first set of self-contained inertial sensors with respect to one or more of the second set of self-contained inertial sensors; and

a substantially known or detectable timing relationship between one or more of the first set of self-contained inertial sensors with respect to one or more of the second self-contained inertial sensors.

60. A method as in claim 56, comprising configuring the first and second set of self-contained inertial sensors with at least one of:

a game, video game, video simulation, a medical device;

a robotic device a toy a communication system and a navigation system.

61. A method as in claim 56, wherein the model is responsive to changes in either position or orientation of a movable controller in a three-dimensional space.

62. A method as in claim 56, wherein the model includes at least one of:

a game, video game, or video simulation, a medical device, a robotic device, and a toy.

63. A method as in claim 56, providing sensors include other than self-contained inertial sensors.

64. A method as in claim 56, wherein the sensors include at least one of:

an accelerometer, a gyroscope;

a compass, an electromagnetic field sensor, a Hall effect sensor, a Doppler sensor, an optical recognizer, and a sonic device.

65. A method as in claim 56, comprising presenting one or more images in response to the model, and the sensors.

66. A method as in claim 65, wherein the step of presenting is responsive to a substantially known or detectable configuration of the device with respect to the sensors.

67. An apparatus for controlling a character in a video game comprising:

a first device being operable independently and including a first set of less than six self-contained inertial sensors, the inertial sensors being insufficient to track changes in both position and orientation of the device in a three-dimensional space; and

a second device including a second sets of less than six self-contained inertial sensors, the second device removably attached to the first device, outputs from the second set of the inertial sensors being coupled via a wireless link to the first device;

elements restricting that first device and second device into a defined configuration, the defined configuration providing information to determine a set of restrictions of movement of the first set of self-contained inertial sensors and the second sets of self-contained inertial sensors with respect to each other,

wherein, a combination of a first time series responsive to the first set of self-contained inertial sensors and a second time series responsive to at least one of the second sets of self-contained inertial sensors includes sufficient information to track changes in both position and orientation of the apparatus in the three-dimensional space.

68. An apparatus as in claim 67, including a model, the model being responsive to changes in either position or orientation of an object in a three-dimensional space.

69. An apparatus as in claim 68, wherein one or more of the self-contained inertial sensors in the first set and second set is substantially movable in at least one of position or orientation by a human motive force.

70. An apparatus as in claim 68, wherein the object is at least one of:

handheld, wearable, attachable to a living creature.

71. An apparatus as in claim 68, wherein the object is configurable with at least one of:

a game, video game, video simulation, a medical device, a robotic device, a toy, a communication system, and a navigation system.

72. An apparatus as in claim 67, including

a combiner responsive to (a) a first time series responsive to one or more of the first set of sensors, and (b) a second time series responsive to one or more of the second set of sensors.

73. An apparatus as in claim 72, wherein the combiner is responsive to at least one of:

a substantially known or detectable configuration of one or more of those first sensors with respect to one or more of those second sensors; and

a substantially known or detectable timing relationship between one or more of those first sensors with respect to one or more of those second sensors.

74. An apparatus for controlling a character in a video game, comprising:

a first set of self-contained inertial sensors enclosed in a first device being insufficient by itself to concurrently track substantially unconstrained changes in both position and orientation of the first device in a three-dimensional space; and

a second sets of self-contained inertial sensors enclosed in a second device being insufficient by itself to concurrently track substantially unconstrained changes in both the position and the orientation of the second device in a three-dimensional space;

elements restricting the first device and the second device into a defined configuration, signals from the second device being coupled to the first device that is configured to correlate the signals with signals from the first set of inertial sensors and that defined configuration providing information to determine a set of restrictions of movement of that first set of sensors and those second sets of sensors with respect to each other,

wherein a combination of a first time series responsive to the first set of sensors and a second time series responsive to at least one of the second sets of sensors, includes sufficient information to concurrently track substantially unconstrained changes in both position and orientation of the apparatus in a three-dimensional space.

75. An apparatus as in claim 74, including

a model, that model being responsive to changes in either position or orientation of that object in a three-dimensional space.

76. An apparatus as in claim 74, wherein one or more of the sensors is substantially movable in at least one of position or orientation by a human motive force.

77. An apparatus as in claim 74, wherein the apparatus is at least one of:

handheld, wearable, attachable to a living creature.

78. An apparatus as in claim 74, wherein the apparatus is configurable with at least one of:

a game, video game, or video simulation, a medical device, a robotic device, a toy, a communication system, and a navigation system.

79. An apparatus as in claim 74, including a combiner responsive to (a) a first time series responsive to one or more of the first set of sensors, and (b) a second time series responsive to one or more of the second set of sensors.

80. An apparatus as in claim 79, wherein the combiner is responsive to at least one of:

a substantially known or detectable configuration of one or more of the first sensors with respect to one or more of the second sensors; and

a substantially known or detectable timing relationship between one or more of the first sensors with respect to one or more of the second sensors.

81. An apparatus for controlling a character in a video game, comprising:

a first set of less than six first self-contained inertial sensors enclosed in a device independently operable and the device being augmented by attaching a case thereto, the case including a second set of less than six second self-contained inertial sensors, a total number of the first and second self-contained inertial sensors being at least six, those first and second contained inertial sensors collectively providing a time series of information being sufficient information to track changes in both position and orientation of an object in a three-dimensional space; and

a model of a possibly-fictional world being controlled by the device with the case.

82. An Apparatus as in claim 81, wherein the model includes at least one of:

a game, video game, or video simulation;

a medical device, a robotic device and a toy.

83. An apparatus as in claim 81, wherein the apparatus is substantially movable in at least one of position or orientation by a human motive force unassisted by a mechanical device.

84. An apparatus as in claim 81, wherein the apparatus is at least one of:

handheld, wearable, attachable to a living creature.

85. An apparatus as in claim 81, wherein the apparatus is configurable with at least one of:

a game, video game, video simulation, a medical device, a robotic device, a toy, a communication system, and a navigation system.

86. An apparatus as in claim 81, including a combiner responsive to (a) a first time series responsive to one or more of the first set of sensors, and (b) a second time series responsive to one or more of the second self-contained inertial sensors.

87. An apparatus as in claim 86, wherein the combiner is responsive to at least one of:

a substantially known or detectable configuration of one or more of those first sensors with respect to one or more of those second sensors; and

a substantially known or detectable timing relationship between one or more of those first self-contained inertial sensors with respect to one or more of those second self-contained inertial sensors.

88. An apparatus as in claim 81, including sensors other than the self-contained inertial sensors.

89. Apparatus as in claim 81, including a presentation device responsive to the model, the sufficient information, and the first and second self-contained inertial sensors.

90. An apparatus as in claim 89, wherein the presentation device is responsive to a substantially known or detectable configuration of the device with respect to the first and second self-contained inertial sensors.

91. An apparatus as in claim 81, wherein at least one of the the first and second self-contained inertial sensors includes at least one of:

an accelerometer, a gyroscope, a compass, an electromagnetic field sensor, a Hall effect sensor, a Doppler sensor, an optical recognizer, and a sonic device.

92. An apparatus as in claim 81, wherein the model is responsive to changes in either position or orientation of the object in the three-dimensional space.

93. An apparatus comprising:

a set of first self-contained inertial sensors collectively providing a first time series of information, the first time series being insufficient by itself to determine at least one of location or orientation of a first device in a three-dimensional space;

a set of second self-contained inertial sensors collectively providing a second time series of information, the second time series being insufficient to determine at least one of location or orientation of a second device in a three-dimensional space, wherein the first and second devices are coupled electronically and enclosed in the apparatus; and

a combiner of that first time series and that second time series, wherein an output of that combiner includes sufficient information to track changes in both position and orientation of the apparatus in a three-dimensional space.

94. An apparatus as in claim 93, wherein that combiner is responsive to at least one of

a substantially known or detectable configuration of one or more of the first sensors with respect to one or more of the second sensors;

and

a substantially known or detectable timing relationship between one or more of the first sensors with respect to one or more of the second sensors.

95. An apparatus comprising:

a plurality of inertial sensors providing sensor data sufficient to derive six relative linear and angular motions of the apparatus as the apparatus is caused by the user to move from one location to another location in a three-dimensional space, wherein a processor is configured to correlate the sensor data from the inertial sensors to determine relative positions and orientations of the apparatus with respect to a reference point so that a trajectory of the apparatus is inferred and used to render a corresponding movement in a display,

wherein the apparatus is an integrated controller including a first device and a second device, the first device, capable of operating independently, comprises a transceiver; and

the second device, detachably coupled to the first device, comprises some or all of the inertial sensors; and

an interface to communicate with the first device to transport some or all of the sensor data from the inertial sensors therein to the first device via the interface so that the first device transports the sensor data, via the transceiver, to a base device that is connected to the display.

96. The apparatus as recited in claim 95, wherein the first device is a basic controller comprising:

a first set including one or more but not all of the inertial sensors, wherein the sensor data from the inertial sensors in the basic controller is insufficient to derive the six relative linear and angular motions of the apparatus;

the second device is an auxiliary device comprising;

a second set including one or more but not all of the inertial sensors, wherein some or all of the sensor data from the inertial sensors in the auxiliary device is transported to the basic controller via the interface so that the sensor data together from the basic controller and the auxiliary device is sufficient to derive the six relative linear and angular motions of the apparatus.

97. The apparatus as recited in claim 96, wherein the auxiliary device does not operate independently and is designed to be operated with the basic controller, any sensing measurement from the auxiliary device is complementary to the basic controller so that a processor in the basic controller is configured to correlate the sensor data from the inertial sensors in the basic controller, if any, and in the auxiliary device.

98. The apparatus as recited in claim 97, wherein the inertial sensors are substantially self-contained, as a result, there is no restriction on movement and there is no setup or configuration of external sources by the user to operate the apparatus.

99. The apparatus as recited in claim 97, wherein the linear and angular motions of the apparatus control an animation of an object being manipulated in a virtual world.

100. The apparatus as recited in claim 95, wherein the processor is configured to adjust elements of the estimated positions and orientations in response to data from at least another sensor, not being one of the plurality of inertial sensors, that provides information about at least one element of absolute positions and orientations of the apparatus.

101. A physical medium maintaining instructions interpretable by a computing device, the instructions being disposed to direct the computing device to:

detect a first set of self-contained inertial sensors enclosed in a first device, that first device being insufficient by itself to concurrently track substantially unconstrained changes in both the position and the orientation of an object in a three-dimensional space; and

detect a second set of self-contained inertial sensors enclosed in a second device, the second device being insufficient by itself to concurrently track substantially unconstrained changes in both the position and the orientation of an object in a three-dimensional space;

restrict those first and second devices into a defined configuration, that defined configuration providing information to determine a set of restrictions of movement of that first set of sensors and those second sets of sensors with respect to each other,

wherein those instructions to restrict provide sufficient information, that a combination of a first time series responsive to that first set of sensors and a second time series responsive to at least one of those second sets of sensors, includes sufficient information to concurrently track substantially unconstrained changes in both position and orientation of that object in a three-dimensional space.

102. A physical medium as in claim 101, including instructions disposed to direct the computing device to provide, in response to a set of tracked changes in both position and orientation of the object in a three-dimensional space.

103. A physical medium as in claim 102, wherein the instructions to provide a set of tracked changes include instructions to combine a first time series responsive to one or more of the first set of sensors with a second time series responsive to one or more of the second set of sensors.

104. A physical medium as in claim 103, wherein the instructions to combine direct the computing device to operate responsive to at least one of

a substantially known or detectable configuration of one or more of those first sensors with respect to one or more of those second sensors;

and

a substantially known or detectable timing relationship between one or more of those first sensors with respect to one or more of those second sensors,

at least one of (a) that substantially known or detectable configuration, (b) that substantially known or detectable timing relationship, being responsive to those instructions to restrict.

105. A physical medium maintaining instructions interpretable by a computing device, the instructions being disposed to direct the computing device to:

augment a device by attaching removably a case thereto, the device containing a first set of less than six self-contained inertial sensors, the device being insufficient by itself to concurrently track substantially unconstrained changes in both the position and the orientation of an associated object in a three-dimensional space;

the instructions to augment including instructions disposed to direct the computing device to

provide a second set of self-contained inertial sensors enclosed in a case;

receive in the device signals from the second set of inertial sensors in the case, and provide an augmented time series in response to those first and second sets of sensors, the augmented time series including sufficient information to concurrently track substantially unconstrained changes in both position and orientation of the associated object in a three-dimensional space.

106. A physical medium as in claim 105, including instructions disposed to direct that computing device to

provide, in response to that sufficient information, a set of tracked changes in both position and orientation of that object in a three-dimensional space.

107. A physical medium as in claim 106, wherein

those instructions to provide a set of tracked changes include instructions to combine a first time series responsive to one or more of that first set of sensors with a second time series responsive to one or more of that second set of sensors.

108. A physical medium as in claim 107, wherein

those instructions to combine direct that computing device to operate responsive to at least one of

a substantially known or detectable configuration of one or more of those first sensors with respect to one or more of those second sensors;

and

a substantially known or detectable timing relationship between one or more of those first sensors with respect to one or more of those second sensors.

109. A physical medium as in claim 105, wherein

those instructions to provide an augmented time series include instructions to combine a first time series responsive to one or more of that first set of sensors with a second time series responsive to one or more of that second set of sensors.

110. A physical medium maintaining instructions interpretable by a controlling device, the instructions being disposed to direct the controlling device to:

generate sensing signals from at least six self-contained inertial sensors enclosed therein, wherein among the at least six self-contained inertial sensors there must be three for linear acceleration along three linear axes and three for determining angular motion;

provide an augmented time series in response to the inertial sensors, the augmented time series including sufficient information to concurrently track substantially unconstrained changes in both position and orientation of the controlling device in a three-dimensional space, wherein the controlling device is configurable with at least one of;

a game, video game, or video simulations a medical device, a robotic device, a toy, a communication system, and a navigation system.

111. A physical medium as in claim 110, including instructions disposed to direct the controlling device to provide, in response to the sufficient information, a set of tracked changes in both position and orientation of the controlling device in a three-dimensional space.