Identifying a 3-D motion on 2-D planes

• US 9,158,380 B2
• Filed: 07/09/2012
• Issued: 10/13/2015
• Est. Priority Date: 07/09/2012
• Status: Expired due to Fees

• Current Assignee: Wickr Inc. (Amazon.com, Inc.)
• Original Assignee: Net Power & Light, Inc (Amazon.com, Inc.)
• Inventors: Dubovik, Gleb
• Primary Examiner(s): Chang, Kent
• Assistant Examiner(s): Shah, Sujit
• Application Number:
  US13/544,930
• Publication Number:
  US 20140009256A1
• Time in Patent Office: 1,191 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/0354 with detection of 2D relative movements between the device, or an operating part thereof, and a plane or surface, e.g. 2D mice, trackballs, pens or pucks
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus

Abstract

Systems and methods for recognizing 3-D human motions are described. In some embodiments, the present disclosure contemplates a variety of methods and systems for recognizing a 3-D motion on 2-D planes. The 3-D human motions may include, but is not limited to, kinematic based gestures, such as clapping, spell casting, security gestures, etc.

Claims

1. A computer-implemented method for recognizing a specific gesture based on a 3-D motion of a portable device, the computer-implemented method comprising:

obtaining linear acceleration motion data and device orientation data of the portable device in reference to gravity at a plurality of times at fixed interval, the linear acceleration motion data including a linear acceleration vector in reference to the gravity;

wherein the linear acceleration motion data and the device orientation data in reference to the gravity are obtained by one or more accelerometers associated with the portable device;

obtaining relative angular rotation data of the portable device at the plurality of times, the angular rotation data including an angular velocity vector;

wherein the relative angular rotation data is obtained by one or more gyroscopes, or multiple depth cameras associated with the portable device;

defining a plurality of 2-D local planes at each of the plurality of times according to the linear acceleration vector, and the angular velocity vector obtained at each of the respective plurality of times;

analyzing 2-D kinematic data associated with the motion of the portable device on each of the plurality of defined 2-D local planes, the 2-D kinematic data obtained via the one or more accelerometers and the one or more gyroscopes or multiple depth cameras;

obtaining non-kinematic data of the portable device, the non-kinematic data including audio data from an audio input device associated with the portable device;

wherein the audio data is indicative of a predefined gesture;

identifying the 3-D motion by comparing the 2-D kinematic data associated with the motion of the portable device in one or more of the plurality of 2-D local planes;

identifying the specific gesture by analyzing the identified 3-D motion and the non-kinematic data of the portable device; and

performing or initiating a predefined action indicated by the identified specific gesture.

2. The computer-implemented method as recited in claim 1, wherein the one or more accelerometers include a 3-axis accelerometer.

3. The computer-implemented method as recited in claim 1, wherein the relative angular rotation data includes relative degree of rotation data on roll axis, pitch axis, and yaw axis.

4. The computer-implemented method as recited in claim 3, wherein the one or more gyroscopes include a 3-axis gyroscope.

5. The computer-implemented method as recited in claim 3, wherein the portable device further includes one or more magnetometer, the one or more magnetometer configured to obtain absolute yaw rotation of the portable device to a fixed direction.

6. The computer-implemented method as recited in claim 1, further comprising:

generating a 3-D motion pattern of an operator of the portable device according to multiple distinct inputs of 3-D motions from the operator.

7. The computer-implemented method as recited in claim 6, wherein the 3-D motion pattern is dynamically constructed or expanded by analyzing the operator'"'"'s 3-D motion pattern over time using predefined machine learning algorithms.

8. The computer-implemented method as recited in claim 7, further comprising:

storing the operator'"'"'s 3-D motion pattern on the portable device, other computing devices of the operator, and/or a cloud computing service.

9. The computer-implemented method as recited in claim 1, further comprising:

providing an operator of the portable device a set of predetermined functions of 3-D motion recognition sensitivity according to the operator'"'"'s physical characteristics.

10. The computer-implemented method as recited in claim 9, wherein the portable device includes the audio input device.

11. The computer-implemented method as recited in claim 1, wherein the portable device is a wireless controller, a personal digital assistant (PDA), a smart phone, a tablet, a Window/Mac laptop, a Linux machine, a netbook, a cloudbook, or a media player.

12. A system for recognizing a specific gesture based on a 3-D motion, the system comprising:

a portable device, the portable device including;

one or more processors;

one or more accelerometers;

one or more gyroscopes;

one or more audio input devices; and

one or more memory units with instructions stored thereon, which when executed by the one or more processors cause the system to;

obtain linear acceleration motion data and device orientation data of the portable device in reference to gravity at a plurality of times at fixed interval, the linear acceleration motion data including a linear acceleration vector in reference to the gravity;

wherein the linear acceleration motion data and the device orientation data in reference to the gravity are obtained by the one or more accelerometers;

obtain relative angular rotation data of the portable device, the angular rotation data including an angular velocity vector;

wherein the relative angular rotation data is obtained by the one or more gyroscopes;

define a plurality of 2-D local planes at each of the plurality of times according to the linear acceleration vector and the angular velocity vector;

analyze 2-D kinematic data associated with the motion of the portable device on each of the plurality of defined 2-D local planes, the 2-D kinematic data obtained via the one or more accelerometers and the one or more gyroscopes;

obtain non-kinematic data of the portable device, the non-kinematic data including audio data from the one or more audio input device associated with the portable device;

wherein, the audio input data is indicative of a predefined gesture;

identify the 3-D motion by comparing the 2-D kinematic data associated with the motion of the portable device in one or more of the plurality of 2-D local planes;

identify the specific gesture by analyzing the identified 3-D motion and the kinematic data of the portable device; and

perform or initiate a predefined action indicated by the identified specific gesture.

13. The system as recited in claim 12, wherein the one or more accelerometers include a 3-axis accelerometer.

14. The system as recited in claim 12, wherein the relative angular rotation data includes relative degree of rotation data on roll axis, pitch axis, and yaw axis.

15. The system as recited in claim 14, wherein the one or more gyroscopes include a 3-axis gyroscope.

16. The system as recited in claim 14, wherein the portable device further includes one or more magnetometer, the one or more magnetometer configured to obtain absolute yaw rotation of the portable device to a fixed direction.

17. The system as recited in claim 12, wherein the one or more memory units have further instructions stored thereon, which when executed by the one or more processors cause the system to further:

generate a 3-D motion pattern of an operator of the portable device according to multiple distinct inputs of 3-D motions from the operator.

18. The system as recited in claim 17, wherein the 3-D motion pattern is dynamically constructed or expanded by analyzing the operator'"'"'s 3-D motion pattern over time using predefined machine learning algorithms.

19. The system as recited in claim 18, wherein the one or more memory units have further instructions stored thereon, which when executed by the one or more processors cause the system to further:

store the operator'"'"'s 3-D motion pattern on the portable device, other computing devices of the operator, and/or a cloud computing service.

20. The system as recited in claim 12, wherein the one or more memory units have further instructions stored thereon, which when executed by the one or more processors cause the system to further:

provide an operator of the portable device a set of predetermined functions of 3-D motion recognition sensitivity according to the operator'"'"'s physical characteristics.

21. The system for recognizing a specific gesture based on a 3-D motion as recited in claim 12, wherein the portable device is a wireless controller, a personal digital assistant (PDA), a smart phone, a tablet, or a laptop.