IDENTIFYING A 3-D MOTION ON 2-D PLANES

• US 20140009256A1
• Filed: 07/09/2012
• Published: 01/09/2014
• Est. Priority Date: 07/09/2012
• Status: Active Grant

• Current Assignee: Wickr Inc. (Amazon.com, Inc.)
• Original Assignee: Net Power & Light, Inc (Amazon.com, Inc.)
• Inventors: Dubovik, Gleb
• Granted Patent: US 9,158,380 B2
• Time in Patent Office: Days
• US Class Current: 340/4.31
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/0354 with detection of 2D relative movements between the device, or an operating part thereof, and a plane or surface, e.g. 2D mice, trackballs, pens or pucks
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus

Abstract

Systems and methods for recognizing 3-D human motions are described. In some embodiments, the present disclosure contemplates a variety of methods and systems for recognizing a 3-D motion on 2-D planes. The 3-D human motions may include, but is not limited to, kinematic based gestures, such as clapping, spell casting, security gestures, etc.

Claims

1. A computer-implemented method for recognizing a 3-D motion of a portable device, the computer-implemented method comprising:

obtaining linear acceleration motion data and device orientation data of the portable device in reference to gravity, the linear acceleration motion data including a linear acceleration vector in reference to the gravity;

obtaining relative angular rotation data of the portable device, the angular rotation data including an angular velocity vector;

determining a 2-D local plane according to the linear acceleration motion data, the device orientation data, the relative angular rotation data, and/or the direction of gravity; and

identifying the 3-D motion by analyzing the linear acceleration motion data, the device orientation data, and the relative angular rotation data in the 2-D local plane.

2. A computer-implemented method as recited in claim 1, wherein the linear acceleration motion data and the device orientation data in reference to the gravity are obtained by one or more accelerometers, the one or more accelerometers including a 3-axis accelerometer.

3. A computer-implemented method as recited in claim 2, wherein the relative angular rotation data is obtained by one or more gyroscopes, or multiple depth cameras, the relative angular rotation data including relative degree of rotation data on roll axis, pitch axis, and yaw axis.

4. A computer-implemented method as recited in claim 3, wherein the one or more gyroscopes include a 3-axis gyroscope.

5. A computer-implemented method as recited in claim 3, wherein the portable device further includes one or more magnetometer, the one or more magnetometer configured to obtain absolute yaw rotation of the portable device to a fixed direction.

6. A computer-implemented method as recited in claim 1, further comprising:

generating a 3-D motion pattern of an operator of the portable device according to multiple distinct inputs of 3-D motions from the operator.

7. A computer-implemented method as recited in claim 6, wherein the 3-D motion pattern is dynamically constructed or expanded by analyzing the operator'"'"'s 3-D motion pattern over time using predefined machine learning algorithms.

8. A computer-implemented method as recited in claim 7, further comprising:

storing the operator'"'"'s 3-D motion pattern on the portable device, other computing devices of the operator, and/or a cloud computing service.

9. A computer-implemented method as recited in claim 1, further comprising:

performing or initiating a predefined action indicated by the identified 3-D motion.

10. A computer-implemented method as recited in claim 9, further comprising:

providing an operator of the portable device a set of predetermined functions of 3-D motion recognition sensitivity according to the operator'"'"'s physical characteristics.

11. A computer-implemented method as recited in claim 10, wherein the portable device includes an audio input device.

12. A computer-implemented method as recited in claim 11, further comprising:

obtaining audio data from the audio input device; and

identifying the 3-D motion by analyzing the linear acceleration motion data, the device orientation data, and the relative angular rotation data in the 2-D local plane, and the audio data.

13. A computer-implemented method as recited in claim 1, wherein the portable device is a wireless controller, a personal digital assistant (PDA), a smart phone, a tablet, a Window/Mac laptop, a Linux machine, a netbook, a cloudbook, or a media player.

14. A system for recognizing a 3-D motion, the system comprising:

a portable device; and

a motion recognition process executing on the system, the motion recognition process operable to;

obtain linear acceleration motion data and device orientation data of the portable device in reference to gravity, the linear acceleration motion data including a linear acceleration vector in reference to the gravity;

obtain relative angular rotation data of the portable device, the angular rotation data including an angular velocity vector;

determine a 2-D local plane according to the linear acceleration motion data, the device orientation data, the relative angular rotation data, and/or the direction of gravity; and

identify the 3-D motion by analyzing the linear acceleration motion data, the device orientation data, and the relative angular rotation data in the 2-D local plane.

15. A system for recognizing a 3-D motion as recited in claim 14, wherein the linear acceleration motion data and the device orientation data in reference to the gravity are obtained by one or more accelerometers, the one or more accelerometers including a 3-axis accelerometer.

16. A system for recognizing a 3-D motion as recited in claim 15, wherein the relative angular rotation data is obtained by one or more gyroscopes, or multiple depth cameras, the relative angular rotation data including relative degree of rotation data on roll axis, pitch axis, and yaw axis.

17. A system for recognizing a 3-D motion as recited in claim 16, wherein the one or more gyroscopes include a 3-axis gyroscope.

18. A system for recognizing a 3-D motion as recited in claim 16, wherein the portable device further includes one or more magnetometer, the one or more magnetometer configured to obtain absolute yaw rotation of the portable device to a fixed direction.

19. A system for recognizing a 3-D motion as recited in claim 14, wherein the motion recognition process is further operable to generate a 3-D motion pattern of an operator of the portable device according to multiple distinct inputs of 3-D motions from the operator.

20. A system for recognizing a 3-D motion as recited in claim 19, wherein the 3-D motion pattern is dynamically constructed or expanded by analyzing the operator'"'"'s 3-D motion pattern over time using predefined machine learning algorithms.

21. A system for recognizing a 3-D motion as recited in claim 20, wherein the motion recognition process is further operable to store the operator'"'"'s 3-D motion pattern on the portable device, other computing devices of the operator, and/or a cloud computing service.

22. A system for recognizing a 3-D motion as recited in claim 14, wherein the motion recognition process is further operable to perform or initiate a predefined action indicated by the identified 3-D motion.

23. A system for recognizing a 3-D motion as recited in claim 22, wherein the motion recognition process is further operable to provide an operator of the portable device a set of predetermined functions of 3-D motion recognition sensitivity according to the operator'"'"'s physical characteristics.

24. A system for recognizing a 3-D motion as recited in claim 14, wherein the portable device includes an audio input device.

25. A system for recognizing a 3-D motion as recited in claim 24, wherein the motion recognition process is further operable to obtain audio data from the audio input device;

and identify the 3-D motion by analyzing the linear acceleration motion data, the device orientation data and the relative angular rotation data in the 2-D local plane, and the audio data.

26. An apparatus for recognizing a 3-D motion of a portable device, the portable device including one or more accelerometers and one or more gyroscopes, the computer-implemented method comprising:

means for obtaining linear acceleration motion data and device orientation data of the portable device in reference to gravity, the linear acceleration motion data including a linear acceleration vector in reference to the gravity;

means for obtaining relative angular rotation data of the portable device, the angular rotation data including an angular velocity vector;

means for determining a 2-D local plane according to the linear acceleration motion data, the device orientation data, the relative angular rotation data, and/or the direction of gravity;

and

means for identifying the 3-D motion by analyzing the linear acceleration motion data, the device orientation data, and the relative angular rotation data in the 2-D local plane.

27. A computer-implemented method for recognizing a 3-D motion of a portable device, the computer-implemented method comprising:

obtaining linear acceleration motion data and device orientation data of the portable device in reference to gravity, the linear acceleration motion data including a linear acceleration vector in reference to the gravity;

obtaining relative angular rotation data of the portable device, the angular rotation data including an angular velocity vector;

determining a 2-D local plane according to the linear acceleration vector and the angular velocity vector; and

identifying the 3-D motion by analyzing the linear acceleration motion data, the device orientation data and the relative angular rotation data in the 2-D local plane.

28. A computer-implemented method as recited in claim 27, wherein the linear acceleration motion data and the device orientation data in reference to the gravity are obtained by one or more accelerometers, the one or more accelerometers including a 3-axis accelerometer.

29. A computer-implemented method as recited in claim 28, wherein the relative angular rotation data is obtained by one or more gyroscopes, or multiple depth cameras, the relative angular rotation data including relative degree of rotation data on roll axis, pitch axis, and yaw axis, the one or more gyroscopes including a 3-axis gyroscope.