Method and apparatus for interpreting orientation invariant motion

• US 8,655,622 B2
• Filed: 07/05/2008
• Issued: 02/18/2014
• Est. Priority Date: 07/05/2008
• Status: Active Grant

• Current Assignee: Wei Yen
• Original Assignee: AiLive, Inc.
• Inventors: Funge, John, Yen, Wei, Wright, Ian, Wilkinson, Dana, Tu, Xiaoyuan, Reynolds, Stuart, Powers, III, William Robert, Musick, JR., Charles
• Primary Examiner(s): NGHIEM, MICHAEL P
• Application Number:
  US12/168,126
• Publication Number:
  US 20100004896A1
• Time in Patent Office: 2,054 Days
• Field of Search: 702/141, 702/142, 702150-154, 463/32, 463/36, 463/37, 715/700, 715/757, 715/858, 715/863, 345/156, 345/158, 345/169, 345/419, 345/672, 345/676, 345/679
• US Class Current: 702/153
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/22 Setup operations, e.g. calibration, key configuration or button assignment
• CPC Class Codes: A63F 13/24 Constructional details thereof, e.g. game controllers with detachable joystick handles
• CPC Class Codes: A63F 13/42 by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/92 Video game devices specially adapted to be hand-held while playing
• CPC Class Codes: A63F 2300/1018 Calibration; Key and button assignment
• CPC Class Codes: A63F 2300/1043 being characterized by constructional details
• CPC Class Codes: A63F 2300/105 using inertial sensors, e.g. accelerometers, gyroscopes
• CPC Class Codes: A63F 2300/204 the platform being a handheld device
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors

Abstract

Techniques for interpreting orientation invariant motion are disclosed. Unlike a prior art controller that has a specific physical design to induce or force a user to grip the controller in a consistent way, a disclosed controller does not have such a physical design and allows a user to grip the controller in any way that is comfortable to the user. One or more transformations or rotations are designed to transform or rotate readings from inertial sensors housed in the controller to readings independent from how the controller is being gripped by a user.

Claims

1. A method for interpreting orientation invariant motion using a computing unit, the method comprising:

receiving one or more sets of reference readings;

receiving sets of readings over a time period from a set of inertial sensors housed in a handheld device gripped by a user in one of many orientations as the handheld device is being moved from one location to another location in a 3-dimensional space;

computing a set of rotations of the handheld device from the sets of readings;

computing a representation of a movement of the handheld device from the sets of readings, the set of rotations, and the one or more sets of reference readings, wherein the computing of the representation of the movement includes at least one of;

rotating at least some of the sets of readings, rotating at least some of the reference readings, partially rotating at least some of the sets of readings and partially rotating at least some of the reference readings to readings that are substantially independent of how the handheld device is gripped; and

wherein the reference readings include first and second sets of data, the first set of data is determined from a vector field and pertains to an initial orientation of the handheld device gripped in accordance with a reference grip, and the second set of data is determined from the vector field as the handheld device is gripped by a different user with a different grip, and the representation of the movement is substantially independent of how the handheld device is gripped and the reference readings are taken when the handheld device is detected to have a sudden move.

2. The method as recited in claim 1, wherein the reference readings pertain to an initial direction of the sudden move of the handheld device.

3. The method as recited in claim 2, wherein subsequent readings from the inertial sensors are sampled to be the sets of readings, and the reference readings are taken again after the handheld device has been relatively stationary and is detected to have another sudden move.

4. The method as recited in claim 1, wherein at least one element of the reference readings is determined externally, not from the inertial sensors.

5. The method as recited in claim 1, wherein either one or all of the first and second data are taken again after the handheld device has been relatively stationary for a predefined time.

6. The method as recited in claim 1, wherein the vector field is from gravity or a magnetic field.

7. The method as recited in claim 1, wherein said computing a set of rotations includes comparing the first and second sets of data to derive the set of rotations required to rotate the first set of data to be substantially coincident with the second set of data.

8. The method as recited in claim 1, wherein said computing a set of rotations includes calculating the rotations required to rotate the first and second sets of data to be substantially coincident with a first configuration different from a second configuration corresponding to either one of the first and second sets of data.

9. The method as recited in claim 1, wherein the inertial sensors include at least one of an accelerometer, a gyroscope or a magnetic field sensor.

10. The method as recited in claim 1, wherein the inertial sensors include a camera, an infra-red sensor, an optical recognizer, a sonic device, an electromagnetic field sensor, or a device capable of determining information from triangulation.

11. The method as recited in claim 1, further comprising:

providing a motion recognition module configured to derive a type of motion of the handheld device since one or more sets of the reference readings have been taken; and

generating a motion recognition result from the motion recognition module to assign a predefined label, wherein the motion recognition result is substantially independent of how the handheld device was gripped.

12. The method as recited in claim 1, further comprising:

deriving a trajectory of the handheld device; and

tracking relative changes in positions and orientations of the handheld device that are substantially independent of how the handheld device is being gripped.

13. The method as recited in claim 1, wherein the handheld device does not have a particular physical design that indirectly induces a user to grip the handheld device in a particular way.

14. The method as recited in claim 13, wherein the handheld device is a standalone controller to be gripped in any way that is comfortable to the user.

15. The method as recited in claim 13, wherein the handheld device is part of another portable device.

16. The method as recited in claim 15, wherein the portable device is one of a mobile phone, a magic wand, a light-saber, a microphone, a sword, a whip, a drum-stick, or a baton.

17. The method as recited in claim 13, wherein the handheld device is substantially rotationally symmetrical about at least an axis and to be gripped naturally in any orientation about the axis.

18. The method as recited in claim 17 wherein the device is substantially cylindrical or spherical, and to be gripped naturally in any orientation around an axis.

19. A controller comprising:

a housing having an exterior design that allows a user to grip the controller in any way the user prefers, resulting in many orientations as the controller is being moved from one location to another location in a 3-dimensional space; and

a set of inertial sensors housed in the controller, wherein sets of readings over a time period from the inertial sensors are obtained, and at least some of the sets of readings or at least some of reference readings are rotated to readings that are substantially independent of how the controller is gripped to compute a representation of a movement of the controller, and wherein the movement of the controller is substantially independent of how the controller is gripped, and the reference readings are taken when the controller is detected to have a sudden move, and the reference readings include first and second sets of data, the first set of data is determined from a vector field and pertains to an initial orientation of the controller gripped in accordance with a reference grip, and the second set of data is determined from the vector field as the controller is gripped by a different user with a different grip.

20. The controller as recited in claim 19, wherein the reference readings pertain to an initial direction of the sudden move of the controller.

21. The controller as recited in claim 20, wherein subsequent readings from the inertial sensors are sampled to be the sets of readings, and the reference readings are taken again after the controller has been relatively stationary and is detected to have another sudden move.

22. The controller as recited in claim 19, wherein at least one element of the reference readings is determined externally, not from the inertial sensors.

23. The controller as recited in claim 19, wherein either one or all of the first and second data are taken again after the controller has been relatively stationary for a predefined time.

24. The controller as recited in claim 19, wherein the vector field is from gravity or a magnetic field.

25. The controller as recited in claim 19, wherein each of the rotations is determined by comparing the first and second sets of data to rotate the first set of data to be substantially coincident with the second set of data.

26. The controller as recited in claim 19, wherein each of the rotations is determined by calculating the rotations required to rotate the first and second sets of data to be substantially coincident with a first configuration different from a second configuration corresponding to either one of the first and second sets of data.

27. The controller as recited in claim 19, wherein the inertial sensors include at least one of an accelerometer, a gyroscope or a magnetic field sensor.

28. The controller as recited in claim 19, wherein the inertial sensors include a camera, an infra-red sensor, an optical recognizer, a sonic device, an electromagnetic field sensor, or a device capable of determining information from triangulation.

29. The controller as recited in claim 19, wherein the controller is configured to work with a motion recognition module configured to derive how the controller has been moved since the reference readings are taken;

and a motion recognition result from the motion recognition module is used to match a predefined label, wherein the motion recognition result is substantially independent of how the controller is being gripped.

30. The controller as recited in claim 19, wherein the controller is configured to work with a motion recognition module to derive a trajectory of the controller and track relative changes in positions and orientations of the controller that are substantially independent of how the controller is being gripped.

31. The controller as recited in claim 19, wherein the controller is a standalone game controller to be gripped in any way that is comfortable to the user.

32. The controller as recited in claim 31, wherein the controller is part of another portable device.

33. The controller as recited in claim 32, wherein the portable device is one of a mobile phone, a magic wand, a light-saber, a microphone, a sword, a whip, a drum-stick, or a baton.

34. The controller as recited in claim 19, wherein the controller is substantially rotationally symmetrical about at least an axis and to be gripped naturally in any orientation about the axis.

35. The method as recited in claim 19 wherein the controller is substantially cylindrical or spherical, and to be gripped naturally in any orientation around an axis.