Method and apparatus for interpreting orientation invariant motion

• US 20100004896A1
• Filed: 07/05/2008
• Published: 01/07/2010
• Est. Priority Date: 07/05/2008
• Status: Active Grant

• Current Assignee: Wei Yen
• Original Assignee: AiLive, Inc.
• Inventors: Reynolds, Stuart, Tu, Xiaoyuan, Funge, John, Wright, Ian, Yen, Wei, Wilkinson, Dana, Musick, JR., Charles, Powers, III, William Robert
• Granted Patent: US 8,655,622 B2
• Time in Patent Office: Days
• US Class Current: 702/153
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/22 Setup operations, e.g. calibration, key configuration or button assignment
• CPC Class Codes: A63F 13/24 Constructional details thereof, e.g. game controllers with detachable joystick handles
• CPC Class Codes: A63F 13/42 by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/92 Video game devices specially adapted to be hand-held while playing
• CPC Class Codes: A63F 2300/1018 Calibration; Key and button assignment
• CPC Class Codes: A63F 2300/1043 being characterized by constructional details
• CPC Class Codes: A63F 2300/105 using inertial sensors, e.g. accelerometers, gyroscopes
• CPC Class Codes: A63F 2300/204 the platform being a handheld device
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors

Abstract

Techniques for interpreting orientation invariant motion are disclosed. Unlike a prior art controller (e.g., a WII game controller) that has a specific physical design to induce or force a user to grip the controller in a consistent way, a disclosed controller does not have such a physical design and allow a user to grip the controller in any way that is comfortable to the user (e.g., around an axis z), one or more transformation or rotations are designed to transform or rotate readings from inertial sensors housed in the controller to readings independent from how the controller is being gripped by a user.

Claims

1. A method comprising:

receiving one or more sets of reference readings;

receiving sets of readings over a time period from a set of inertial sensors housed in a handheld device gripped by a user in one of many possible orientations as the handheld device is being moved from one location to another location in a 3-dimensional space;

computing a set of rotations;

computing a representation of a movement of the handheld device from the sets of readings, the set rotations, and the set of reference readings, wherein the computing of the representation of the movement

includes at least one of;

rotating at least some of the sets of readings, or rotating at least some of the reference readings, or partially rotating at least some of the sets of readings and partially rotating at least some of the reference readings; and

wherein the representation of the movement is substantially independent of how the handheld is gripped.

2. The method as recited in claim 1, wherein the reference readings are taken when the handheld device is detected to have a sudden move.

3. The method as recited in claim 2, wherein the reference readings pertain to an initial direction of the sudden move of the handheld device.

4. The method as recited in claim 3, wherein subsequent readings from the inertial sensors are sampled to be the sets of readings, and the reference readings are taken again after the handheld device has been relatively stationary and is detected to have another sudden move.

5. The method as recited in claim 1, wherein at least one element of the reference readings is determined externally, not from the inertial sensors.

6. The method as recited in claim 1, wherein the reference readings include first and second sets of data, the first set of data is determined from a vector field and pertains to an initial orientation of the handheld gripped in accordance with a reference grip, and the second set of data is determined from the vector field as the handheld device is gripped by a different user with a different grip.

7. The method as recited in claim 6, wherein either one or all of the first and second data are taken again after the handheld device has been relatively stationary for a predefined time.

8. The method as recited in claim 6, wherein the vector field is from gravity or a magnetic field.

9. The method as recited in claim 6, wherein said computing a set of rotations includes comparing the first and second sets of data to derive the set of rotations required to rotate the first set of data to be substantially coincident with the second set of data.

10. The method as recited in claim 6, wherein said computing a set of rotations includes calculating the rotations required to rotate the first and second set of data to be substantially coincident with a first configuration different from a second configuration corresponding to either one of the first and second sets of data.

11. The method as recited in claim 1, wherein the inertial sensors include at least one of an accelerometer, a gyroscope or a magnetic field sensor.

12. The method as recited in claim 1, wherein the inertial sensors include (a camera, an infra-red sensor, an optical recognizer, a sonic device, an electromagnetic field sensor, and a device capable of determining information from triangulation.

13. The method as recited in claim 1, further comprising:

providing a motion recognition module configured to derive a type of motion of the handheld device since one or more sets of the reference readings are taken; and

generating a motion recognition result from the motion recognition module to assign a predefined label, wherein the motion recognition result is substantially independently of how the handheld device was gripped.

14. The method as recited in claim 1, further comprising:

deriving a trajectory of the handheld device; and

tracking relative changes in positions and orientations of the handheld device that are substantially independent of how the handheld device is being gripped.

15. The method as recited in claim 1, wherein the handheld device does not have a particular physical design that indirectly induces a user to grip the handheld device in a particular way.

16. The method as recited in claim 15, wherein the handheld device is a stand alone controller to be gripped in any way that is comfortable to the user.

17. The method as recited in claim 15, wherein the handheld device is part of another portable device.

18. The method as recited in claim 17, wherein the portable device is one of a mobile phone, a magic wand, a light-saber, a microphone, a sword, a whip, a drum-stick, or a baton.

19. The method as recited in claim 15, wherein the handheld device is substantially rotationally symmetrical about at least an axis and to be gripped naturally in any orientation about the axis.

20. The method as recited in claim 19 wherein the device is substantially cylindrical or spherical, and to be gripped naturally in any orientation around an axis.

21. A controller comprising:

a housing having an exterior design that allows a user to grip the controller in any way the user prefers, resulting in many possible orientations as the controller is being moved from one location to another location in a 3-dimensional space

a set of inertial sensors housed in the controller, wherein sets of readings over a time period from the inertial sensors are obtained, and at least some of the sets of readings or at least some of reference readings are rotated to compute a representation of a movement of the controller, and wherein the movement of the handheld is substantially independent of how the controller is gripped.

22. The controller as recited in claim 21, wherein the reference readings are taken when the controller is detected to have a sudden move.

23. The controller as recited in claim 22, wherein the reference readings pertain to an initial direction of the sudden move of the controller.

24. The controller as recited in claim 23, wherein subsequent readings from the inertial sensors are sampled to be the sets of readings, and the reference readings are taken again after the controller has been relatively stationary and is detected to have another sudden move.

25. The controller as recited in claim 21, wherein at least one element of the reference readings is determined externally, not from the inertial sensors.

26. The controller as recited in claim 21, wherein the reference readings include first and second sets of data, the first set of data is determined from a vector field and pertains to an initial orientation of the handheld gripped in accordance with a reference grip, and the second set of data is determined from the vector field as the controller is gripped by a different user with a different grip.

27. The controller as recited in claim 26, wherein either one or all of the first and second data are taken again after the handheld device has been relatively stationary for a predefined time.

28. The controller as recited in claim 26, wherein the vector field is from gravity or a magnetic field.

29. The controller as recited in claim 26, wherein each of the rotations is determined by comparing the first and second sets of data to rotate the first set of data to be substantially coincident with the second set of data.

30. The controller as recited in claim 26, wherein each of the rotations is determined by calculating the rotations required to rotate the first and second set of data to be substantially coincident with a first configuration different from a second configuration corresponding to either one of the first and second sets of data.

31. The controller as recited in claim 21, wherein the inertial sensors include at least one of an accelerometer, a gyroscope or a magnetic field sensor.

32. The controller as recited in claim 21, wherein the inertial sensors include a camera, an infra-red sensor, an optical recognizer, a sonic device, an electromagnetic field sensor, a device capable of determining information from triangulation.

33. The controller as recited in claim 21, wherein the controller is configured to work with a motion recognition module configured to derive how the controller has been moved since the reference readings are taken;

and a motion recognition result from the motion recognition module is used to match a predefined label, wherein the motion recognition result is substantially independently of how the controller is being gripped.

34. The controller as recited in claim 21, wherein the controller is configured to work with a module to derive a trajectory of the controller;

and track relative changes in positions and orientations of the controller that are substantially independent of how the controller is being gripped.

35. The controller as recited in claim 21, wherein the controller is a stand alone game controller to be gripped in any way that is comfortable to the user.

36. The controller as recited in claim 35, wherein the controller is part of another portable device.

37. The controller as recited in claim 36, wherein the portable device is one of a mobile phone, a magic wand, a light-saber, a microphone, a sword, a whip, a drum-stick, or a baton.

38. The controller as recited in claim 21, wherein the controller is substantially rotationally symmetrical about at least an axis and to be gripped naturally in any orientation about the axis.

39. The method as recited in claim 21 wherein the controller is substantially cylindrical or spherical, and to be gripped naturally in any orientation around an axis.